nils-herrmann/clean_scirepeval huggingface dataset:
Audio tampering detection using multimodal features

The authenticity verification of a User Generated Audio-Video content relative to a real event can be a very critical task especially when the content is shared on the Internet. Audio-Video files need to be checked in order to verify the origin of the content and the absence of alterations that could have changed their semantic content. The paper presents a multimodal approach for audio tampering detection that analyzes both the audio component and the video component of a recorded video file. The proposed solution estimates the volumetric characteristics of the environment where the multimedia content has been captured both from the video and audio signals. Then, the approach checks the consistency of the environment characteristics estimated from the audio signal with respect to those estimated from video files. The proposed solution proves to be useful in identifying video fakes and bootlegs, although it proves to be useful for the localization of added audio effects in a movie or radio track.
A space domain approach for multiple description video coding

In this paper we address the problem of video transmission over unreliable networks using multiple description (MD) coding approach. Recent literature on MD techniques has shown the success of using motion compensation prediction scheme. Even though these MD systems have proved to be robust to error environment, they are forced to use only two descriptions. For this reason this paper proposes two architectures of MD video coders. The first, called DC-MDVC, is based on the classical MD video codec architecture and the principle contribution is the use of a new MD algorithm based on a spatial domain polyphase down-sampling technique. The second architecture, called IF-MDVC, proposes multi-level scalability generating a flexible number of descriptions. The results of both architectures are interesting and the second architecture, the subject of the current research, appears to perform impressively.
Analyzing notch patterns of head related transfer functions in CIPIC and SYMARE databases

The sensation of elevation in binaural audio is known to be strongly correlated to spectral peaks and notches in HRTFs, introduced by pinna reflections. In this work we provide an analysis methodology that helps us to explore the relationship between notch frequencies and elevation angles in the median plane. In particular, we extract the portion of the HRTF due to the presence of the pinna and we use it to extract the notch frequencies for all the subjects and for all the considered directions. The extracted notch frequencies are then clustered using the K-means algorithm to reveal the relationship between notch frequencies and elevation angles. We present the results of the proposed analysis methodology for all the subjects in the CIPIC and SYMARE HRTFs databases.
A methodology for evaluating the accuracy of wave field rendering techniques

In this paper we propose a methodology for assessing the accuracy of techniques of wave field rendering through loudspeaker arrays. In order to measure the rendered wave field we adopt a solution based on a circular harmonic analysis of the sound field captured by a virtual microphone array. As a result of this analysis stage, we are able to compare the target, the theoretical and the measured wave fields, which may differ due to the non-ideality in the loudspeaker array or in the environment that generates some spurious reverberations. Moreover, in order to quantify the error between target, theoretical and measured wave fields, we define some evaluation metrics, based on RMSE and modal analysis of the acquired wave fields. We show some experimental results on real data.
Combined surface interpolation and object segmentation for automatic 3D scene reconstruction

A common limitation of many techniques for 3D reconstruction from multiple perspective views is the poor quality of the results near the object boundaries. The interpolation process applied to "unstructured" 3D data ("clouds" of non-connected 3D points) plays a crucial role in the global quality of the 3D reconstruction. We present a method for interpolating unstructured 3D data, which is able to perform a segmentation of such data into different data sets that correspond to different objects. The algorithm is also able to perform an accurate localization of the boundaries of the objects. The method is based on an iterative optimization algorithm. As a first step, a set of surfaces and boundary curves are generated for the various objects. Then, the edges of the original images are used for refining such boundaries as best as possible. Experimental results with real data are presented for proving the effectiveness of the proposed algorithm.
A music search engine based on semantic text-based query

Search and retrieval of songs from a large music repository usually relies on added meta-information (e.g., title, artist or musical genre); or on specific descriptors (e.g. mood); or on categorical music descriptors; none of which can specify the desired intensity. In this work, we propose an early example of semantic text-based music search engine. The semantic description takes into account emotional and non-emotional musical aspects. The method also includes a query-by-similarity search approach performed using semantic cues. We model both concepts and musical content in dimensional spaces that are suitable for carrying intensity information on the descriptors. We process the semantic query with a Natural Language parser to capture only the relevant words and qualifiers. We rely on Bayesian Decision theory to model concepts and songs as probability distributions. The resulted ranked list of songs are produced through a posterior probability model. A prototype of the system has been proposed to 53 subjects for evaluation, with good ratings on performance, usefulness and potential.
A plenacoustic approach to acoustic signal extraction

This paper considers the problem of separation of acoustic sources from convolutive mixtures captured by a microphone array. The problem is approached through Plane Wave Decomposition (PWD) of the sound field measured at multiple points along the extension of the array. The directional components of the sound field are analyzed by means of the plenacoustic framework to accurately estimate the direction of arrival of the desired and undesired sources at every point at which the PWD is measured. Multiple spatial filters are designed, one for each PWD measurement point, to leave undistorted the desired source and attenuate the interferer. A successive stage of delay and sum of the outputs of the individual spatial filters enables the reconstruction of the desired source. The use of the plenacoustic framework allows us to gather intuitive and immediate interpretation of the acoustic scene. We prove the effectiveness of the proposed solution through simulations on speech data.
A robust data-independent near-field beamformer for linear microphone arrays

This manuscript presents a robust data-independent near-field beamformer using a linear microphone array. We consider a scenario with a desired sound source and an interferer; our goal in this manuscript is to provide a technique to extract the desired source signal while attenuating the interferer. We formulate the beamformer design problem as a convex optimization problem with additional constraints that aim at controlling the spatial response of the beamformer, with emphasis on the robustness of the approach to errors in the localization of the interferer sound source. We validate our approach through simulations.
P2CA: How Much Face Information is Needed?

Multimodal 2D+3D face biometrics commonly report that performance improves relative to that of a single modality. Complete 2D and 3D data can be available during training because they are acquired in a controlled scenario. However, in the evaluation scenario, only partial 2D and 3D data can be acquired and hence available for recognition. In this paper we present experimental results that determine how partial data contribute to the task of recognition using partial principal component analysis (P2CA) algorithm in a multimodal scheme. From our results it seems that discrimination power on individuals is ascribed to different regions of the face if we consider 2D or 3D data.
Mixed 2D-3D Information for Pose Estimation and Face Recognition

Face recognition based on 3D techniques is a promising approach since it takes advantage of the additional information provided by depth which makes the whole approach more robust against illumination and pose variations. However, these 3D approaches require the cooperation of the person to acquire accurate 3D data; thus, they are not appropriated for some applications such as video surveillance or restricted area access points where only a 2D face image is disposable. In this paper, a novel approach is presented which takes advantage of 3D data in the training stage but only requires 2D data in the recognition stage. The proposed method can be used for both pose estimation and face recognition. Moreover, the estimation of the pose can be used as side information to improve the performance of the face recognition stage. Experiments have been carried out on the public UPC face database which is composed of a total of 621 face images of several persons taken from different views and illuminations
Rate-Distortion Analysis of Motion-Compensated Interpolation at the Decoder in Distributed Video Coding

This letter analyzes the coding efficiency of distributed video coding (DVC) schemes that perform motion-compensated interpolation at the decoder. The decoder has access only to the key frames, when generating the side information for intermediate frames. Therefore, the true motion field necessary for this operation is not directly available, and the motion vectors must be estimated at the decoder side, thus introducing displacement estimation errors. The accuracy of the motion-compensated interpolation at the decoder depends on several factors: 1, the overall motion complexity; 2, the temporal coherence of the motion field; and 3, the temporal distance between successive key frames. Adopting a state-space model and a Kalman filtering framework, we obtain an estimate of the displacement error variance. This is used to determine the rate-distortion function of the overall coding scheme, that takes into account both intra-coded key frames and DVC-coded frames. The proposed model shows that motion-compensated interpolation is unable to achieve the coding efficiency of conventional motion-compensated predictive coding. In addition, the model provides a good estimate of the group of pictures size that optimizes the coding efficiency. Experimental results on real video sequences validate the results of the proposed model.
Robust area matching

We propose a general and robust solution to the problem of close-range 3D reconstruction of objects from stereo correspondence of luminance profiles. The method does not require a particular camera geometry, and can be implemented with an arbitrary number of CCD cameras. Its robustness can be mainly attributed to the physicality of the matching process, which is performed in the 3D space, while taking both geometric and radiometric distortions into account. Extensive tests have been performed over a variety of real scenes, using a calibrated trinocular camera system.
A linear operator for the computation of soundfield maps

In the process of soundfield imaging, as defined in the literature, a microphone array is subdivided into overlapping sub-arrays and soundfield images are obtained by juxtaposition of spatial spectra computed from individual subarray data. In this paper we show that the whole process can be conveniently seen as a linear transformation applied to array data. This linear transformation embeds a nonlinear mapping to cast the directional information in a more convenient domain: the ray space. We show by simulations that the proposed formulation is suitable for fast implementation of the soundfield imaging operation, and, more specifically, for the localization of acoustic sources.
Efficient source localization and tracking in reverberant environments using microphone arrays

In this paper, we propose an algorithm for acoustic source localization and tracking that is suitable for reverberant environments. The approach that we propose is based on the iterative identification of the FIR channels that link source and microphones through an LMS method (multi-channel LMS), but we propose additional solutions that significantly improve this method in terms of computational efficiency and localization reliability, without affecting its convergence properties. This is achieved through a modified block-wise implementation of the approach combined with Kalman filtering. We also show the results of extensive comparative testing using novel performance parameters for the assessment of localization reliability.
3-D Body Posture Tracking For Human Action Template Matching

In this paper we present a novel approach to 3-D human action classification based on the analysis of volumetric data obtained form the joint processing of video sequences acquired by a multiple-camera system. The use of volumetric data makes the system very robust and avoids problems related the typical human body self-occlusions and motion ambiguities, very common in an independent camera-by-camera analysis. A shape descriptor of a human body is obtained in order to capture only posture-dependent characteristics and its outputs at each time instant are collected together in action feature matrices. The use of dynamic time warping approach for action template matching accounts for possible temporal nonlinear distortions among different instances of the same gesture and allows gesture classification
Embedding indexing information in audio signal using watermarking technique

We propose a watermarking scheme applied to digital audio signals. The goal of the project is to hide information which can identify both the tracks and the user who requests them. This technique is used for automatic identification of audio signals in broadcast programs. There are two main features that have to be satisfied: inaudibility and robustness to common editing operations. The algorithm works in a hybrid way: the watermark masking is in the frequency domain while the insertion is in the time domain. This approach allows detection to be performed directly in the time domain. In order to insert the watermark, the signal is split into frames, each of which hides binary information inserted using a spread spectrum sequence. To improve robustness, we use an error correcting code which allows the reduction of error probability. We discuss first the watermark embedding system, then the detection approach and finally the main results are presented.
Musical audio semantic segmentation exploiting analysis of prominent spectral energy peaks and multi-feature refinement

In this paper we present a novel hierarchical and scalable three-stage algorithm to effectively perform musical audio semantic segmentation. In the first stage, the energy spectrum of the entire audio track is analyzed to find significant energy textures that may characterize different semantic segments; in the second and third stages, tonal and timbric features are used to refine the segmentation by moving or deleting segment boundaries. Experimental results on a set of 58 songs show that our algorithm is able to attain good semantic segmentation just after the first step, with a precision of 64% and a recall of 96%. After second step the precision increases to 79%; the best precision result is obtained after the third step, where a value of 85% is reached. In this step the minimum average recall value of 92% is obtained.
Resolution issues in soundfield imaging: A multiresolution approach to multiple source localization

Soundfield imaging is a novel technique for efficiently encoding, representing and processing the wave field captured by a microphone array. Some of the possible applications include source and reflector localization. In this manuscript we discuss the resolution properties of the soundfield image and the trade-offs that need to be considered while designing the “soundfield camera”. Then we propose a source localization algorithm that exploits multiple sound-field images of the same acoustic scene, acquired with different resolutions, to simultaneously localize acoustic sources with widely different distances from the array.
Localization of Acoustic Sources Through the Fitting of Propagation Cones Using Multiple Independent Arrays

In this paper, we propose a novel acoustic source localization method that accommodates the general scenario of multiple independent microphone arrays. The method is based on a 3-D parameter space defined by the 2-D spatial location of a source and the range difference extracted from the time difference of arrival (TDOA). In this space, the set of points that correspond to a given range lie on a circle that expand as the range increases, forming a cone whose apex is the actual location of the source. In this parameter space, the lack of synchronization between arrays results in the fact that clusters of data associated to individual arrays are free to shift along the range axis. The cone constraint, in fact, enables the realignment of such clusters while positioning the cone vertex (source location), thus resulting in a joint data re-synchronization and source localization. We also propose a novel and general analysis methodology for swiftly assessing the localization error as a function of the TDOA uncertainties, which is remarkably accurate for small localization bias. With the aid of this method, simulations and experiments on real data, we show that the cone-fitting process offers excellent localization accuracy in the scenario of multiple unsynchronized arrays, as well as in simpler single-array scenarios, also in comparison with state-of-the-art techniques. We also show that the proposed method offers the desired flexibility for adapting to arbitrary geometries of microphone clusters.
Phylogenetic analysis of near-duplicate and semantically-similar images using viewpoint localization

The need of reliable authentication and forensic analysis tools for multimedia contents has recently led to the investigation of phylogenetic analysis strategies for near-duplicate (ND) images, i.e., pictures generated from a common one through processing operations. As an example, it is possible to detect which image within a group has been used to generate the others. Unfortunately, the accuracy of these algorithms is significantly impaired when the analyzed set includes semantically-similar (SS) images, i.e., pictures reproducing the same subject from different viewpoints that can be easily confused as NDs. This paper presents an image phylogenetic analysis strategy that is able to distinguish between SS and ND pictures thanks to the estimation of the geometric localization of the acquiring viewpoints. The proposed solution has been tested under different experimental setups showing an improved accuracy and a lower computational load with respect to other state-of-the-art strategies.
High resolution imaging of acoustic reflections with spherical microphone arrays

This paper proposes a methodology for the accurate visualization of acoustic reflections in a room from acoustic measurements by a spherical microphone array. The goal is to provide insight on the relationship between architectural and acoustic features. This task requires high-resolution acoustic images. In this contribution, we achieve this goal by introducing two main modifications of existing approaches. At first, we adopt an explicit model that takes into account the scattering due to the rigid spherical surface where the microphone capsules are hosted. Then, we obtain an estimate of the acoustic power coming from a grid of directions using a spectral analysis approach based on the matching of array data covariance matrix. We conclude this manuscript by showing applications of the devised methodology in real world cases.
Tracking Multiple Acoustic Sources in Reverberant Environments using Regularized Particle Filter

This paper concerns the problem of tracking acoustic sources in reverberant environments by using a particle filter. The localization problem is transformed into the retrieval of the unobservable state of a dynamical model through noisy measures. Though effective, two problems are related to particle filter: the degeneracy phenomenon (all particles but one are not significative) and the loss of diversity (all particles collapse on the same point). By using Regularized par ticle filter (RPF) and Expectation Maximization (EM) we propose a solution to both problems. Experimental results validate the pro posed solution: Regularized Particle Filter enables to obtain a RMS error lower than 0.2m with a reverberation time of 0.6s.
The ORIGAMI Project: Advanced tools for creating and mixing real and virtual content in film and TV production

The goal of the EC-funded IST project ORIGAMI was the development of advanced tools and new production techniques for high-quality and seamless integration of real and virtual content in film and TV productions. In particular, the project focused on pre-production tools for automatic virtual set creation through image-based 3-D modelling of environments and objects. One key goal of the project was to achieve real-time in-studio pre-visualisation of the virtual elements (objects and actors) of the extended set. In this contribution the studio pre-visualisation system that has been developed for the project is illustrated, and its usage within a high-end film and TV production environment is described. Furthermore an overview of the developed solutions for automatic generation of 3-D models of environments, static objects and (moving) actors is given.
Multicamera rig calibration by double-sided thick checkerboard

A multi-camera rig calibration algorithm based on a double sided planar target is proposed. Due to their inherently simple realisation, low cost and accuracy, planar calibration targets came out as one of the most largely adopted calibration tools both for intrinsic and extrinsic camera parameters. However, concerning the estimation of extrinsic parameters, one of the major drawbacks of these targets is their requirement for distinct target visibility from both cameras. This prevents many configurations from being adopted where, e.g. two cameras are facing each other. An inexpensive solution could be based on printing/pasting a planar pattern on both target sides, however, the relative misalignment between the patterns on the two sides and the target thickness could be unknown. The authors propose a solution where double-sided target displacement error is estimated together with the extrinsic parameters allowing the reuse of all the available planar calibration tools in less constrained configurations. To assess their approach the authors tested the system in two scenarios, one using two professional 4K cameras and one using two smartphones.
Audio phylogenetic analysis using geometric transforms

Whenever a multimedia content is shared on the Internet, a mutation process is being operated by multiple users that download, alter and repost a modified version of the original data leading to the diffusion of multiple near-duplicate copies. This effect is also experienced by audio data (e.g., in audio sharing platforms) and requires the design of accurate phylogenetic analysis strategies that permit uncovering the processing history of each copy and identify the original one. This paper proposes a new phylogenetic reconstruction strategy that converts the analyzed audio tracks into spectrogram images and compare them using alignment strategies borrowed from computer vision. With respect to strategies currently-available in literature, the proposed solution proves to be more accurate, does not require any a-priori knowledge about the operated transformations, and requires a significantly-lower amount of computational time.