nils-herrmann/clean_scirepeval huggingface dataset:
Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues

This paper introduces a novel contextual model for the recognition of people's visual focus of attention (VFOA) in meetings from audio-visual perceptual cues. More specifically, instead of independently recognizing the VFOA of each meeting participant from his own head pose, we propose to jointly recognize the participants' visual attention in order to introduce context-dependent interaction models that relate to group activity and the social dynamics of communication. Meeting contextual information is represented by the location of people, conversational events identifying floor holding patterns, and a presentation activity variable. By modeling the interactions between the different contexts and their combined and sometimes contradictory impact on the gazing behavior, our model allows us to handle VFOA recognition in difficult task-based meetings involving artifacts, presentations, and moving people. We validated our model through rigorous evaluation on a publicly available and challenging data set of 12 real meetings (5 hours of data). The results demonstrated that the integration of the presentation and conversation dynamical context using our model can lead to significant performance improvements.
Adaptive motion-compensated wavelet filtering for image sequence coding

This paper deals with new advances made in the field of discrete spatio-temporal filters applied to digital image sequences. Within time-varying images, the temporal correlation of the information is folded within the spatio-temporal domain by motions originating from both camera and object displacements. The spatio-temporal video information can be therefore reformulated in terms of motion trajectories and intensity variations along these trajectories. To yield that signal description, the spatio-temporal scenes will be segmented according to motion. Motion-compensated temporal filters have been used as convolutional filters applied along the assumed motion trajectories. Spectral interpretations show the efficiency of motion-compensated filtering for video signals. As a matter of fact, the whole signal analysis performed in this paper also includes a spatial filtering to achieve a complete spatio-temporal (2-D+T) decomposition. Motion-compensated filtering leads to multiresolution applications. It leads to optimum and adaptive signal-to-noise decomposition procedures based on the temporal correlative content. Such properties allow enhancing tasks like temporal interpolation, image sequence smoothing, and restoration. Simulation results are presented in this paper to illustrate the field of image sequence coding.
Comparison of two methods for unsupervised person identification in TV shows

We address the task of identifying people appearing in TV shows. The target persons are all people whose identity is said or written, like the journalists and the well known people, as politicians, athletes, celebrities, etc. In our approach, overlaid names displayed on the images are used to identify the persons without any use of biometric models for the speakers and the faces. Two identification methods are evaluated as part of the REPERE French evaluation campaign. The first one relies on co-occurrence times between overlay person names and speaker/face clusters, and rule-based decisions which assign a name to each monomodal cluster. The second method uses a Conditionnal Random Field (CRF) which combine different types of co-occurrence statistics and pair-wised constraints to jointly identify speakers and faces.
We are not contortionists: Coupled adaptive learning for head and body orientation estimation in surveillance video

In this paper, we deal with the estimation of body and head poses (i.e orientations) in surveillance videos, and we make three main contributions. First, we address this issue as a joint model adaptation problem in a semi-supervised framework. Second, we propose to leverage the adaptation on multiple information sources (external labeled datasets, weak labels provided by the motion direction, data structure manifold), and in particular, on the coupling at the output level of the head and body classifiers, accounting for the restriction in the configurations that the head and body pose can jointly take. Third, we propose a kernel-formulation of this principle that can be efficiently solved using a global optimization scheme. The method is applied to body and head features computed from automatically extracted body and head location tracks. Thorough experiments on several datasets demonstrate the validity of our approach, the benefit of the coupled adaptation, and that the method performs similarly or better than a state-of-the-art algorithm.
Overview of Multiview Video Coding and Anti-Aliasing for 3D Displays

This paper addresses signal processing issues related to coded representation, reconstruction and rendering of multiview video for 3D displays. We provide an overview of standardization efforts for multiview video that are aimed at reducing data rates required to represent the multiview video in compressed form. We then present an anti-aliasing filtering technique that effectively eliminates ghosting artifacts when rendering multiview video on 3D displays. Since high-frequency components of the signal are removed, substantial reductions in the compressed data rate could also be realized. Finally, we discuss the importance of scalability in the context of multiview video coding and suggest a combined anti-aliasing and scalable decoding scheme to minimize decoding resources for a given 3D display.
Towards maximizing the end-user experience

Increasing bandwidth and diverse devices have led to a proliferation of options for the consumer of digital video. The user can easily be overwhelmed by the diversity of devices, bandwidth and content. In our view, maximizing the end-user experience involves first helping the user to easily search for and retrieve desired content, and then delivering the content to him by seamlessly adapting to his device and available bandwidth. The first part is therefore based on semantic criteria while the second part is based on purely signal-based criteria. We propose that scalable content summarization combined with dynamic content adaptation, and unobtrusive processing of user preferences comprise the key technologies for maximizing the end-user experience.
Drift compensation for reduced spatial resolution transcoding: a summary

This paper discusses the problem of reduced-resolution transcoding of compressed video bitstreams. An analysis of drift errors is provided to identify the sources of quality degradation when transcoding to a lower spatial resolution. Two types of drift error are considered: a reference picture error, which has been identified in previous works, and error due to the noncommutative property of motion compensation and down-sampling, which is unique to this work. To overcome these sources of error, four novel architectures are presented. One architecture attempts to compensate for the reference picture error in the reduced resolution, while another architecture attempts to do the same in the original resolution. We present a third architecture that attempts to eliminate the second type of drift error and a final architecture that relies on an intrablock refresh method to compensate for all types of errors. In all of these architectures, a variety of macroblock level conversions are required, such as motion vector mapping and texture down-sampling. These conversions are discussed in detail. Another important issue for the transcoder is rate control. This is especially important for the intra-refresh architecture since it must find a balance between number of intrablocks used to compensate for errors and the associated rate-distortion characteristics of the low-resolution signal. The complexity and quality of the architectures are compared. Based on the results, we find that the intra-refresh architecture offers the best tradeoff between quality and complexity and is also the most flexible.
Frame compatible formats for 3D video distribution

Stereoscopic video will soon be delivered to the home through various channels. To make this feasible for some channels, the representation of the stereo video is modified to accommodate certain constraints on legacy systems. Among the various constraints that must be considered include the capabilities of production equipment and transmission infrastructure, as well as existing receivers and uncompressed digital interfaces between devices within the home. This paper outlines the typical constraints that are encountered in these domains and provides an overview of the various frame-compatible formats that are being considered for distribution of 3D video through such legacy systems. The benefits and drawbacks of these formats are discussed and the current status in various industry forums is reviewed.
Multi-view Video Compression for 3D Displays

Multi-view 3D displays are preferable to other stereoscopic display technologies because they provide autostereoscopic viewing from any viewpoint without special glasses. However, they require a large number of pixels to achieve high image quality. Therefore, data compression is a major issue for this approach. In this paper, we describe a framework for efficient compression of multi-view video streams for multi-view 3D displays. We enhance conventional multi-view compression pipelines with an additional pre-filtering step that band limits the multi-view signal to the display bandwidth. We show that this pre-filtering step leads to increased image quality compared to state-of-the-art multi-view coding at equal bitrate. We present results of an extensive user study that corroborate the benefits of our approach. Our work suggests that any multi-view compression scheme will benefit from our pre-filtering technique.
Constant quality rate allocation for FGS coding using composite R-D analysis

In this correspondence, we propose a constant quality rate allocation algorithm for fine granularity scalability (FGS) coded videos. The rate allocation problem is formulated as a constrained minimization of quality fluctuation. The minimization is solved using a composite rate distortion (R-D) analysis. For a set of video frames, a composite R-D curve is first computed and then used for computing the optimal rate allocation. This algorithm is efficient because it is neither iterative nor recursive. After the composite R-D curve is computed, it can be used for optimal rate allocation of any rate budget. Moreover, the composite R-D curve can be updated efficiently over sliding windows. Experimental results have shown both the effectiveness and the efficiency of the proposed algorithm.
Fusion of multi-angular aerial images based on epipolar geometry and matrix completion

We consider the problem of fusing multiple cloud-contaminated aerial images of a 3D scene to generate a cloud-free image, where the images are captured from multiple unknown view angles. In order to fuse these images, we propose an end-to-end framework incorporating epipolar geometry and low-rank matrix completion. In particular, we first warp the multi-angular images to single-angle ones based on the estimated fundamental matrices that relate the multi-angular images according to their projective relations to the 3D scene. Then we formulate the fusion process of the warpped images as a low-rank matrix completion problem where each column of the matrix corresponds to a vectorized image with missing entries corresponding to cloud or occluded areas. Results using DigitalGlobe high spatial resolution images demonstrate that our algorithm outperforms existing approaches.
Privacy-preserving approximation of L1 distance for multimedia applications

Alice and Bob possess sequences x and y respectively and would like to compute the l1 distance, namely || x — y ||1 under privacy and communication constraints. The privacy constraint requires that Alice and Bob do not reveal their data to each other. The communication constraint requires that they accomplish the secure distance calculation with a small number of protocol transmissions and key exchanges. This paper describes and analyzes a privacy-preserving approximation protocol for the l1 distance that keeps the communication overhead manageable by performing a Johnson-Lindenstrauss embedding into the l1 space. Then, it performs secure two-party computation of l1 distances using Paillier homomorphic encryption. The protocol is implemented for private querying of face images, while maintaining a low communication overhead between the querying party and a remote database of face feature vectors.
Toward realtime side information decoding on multi-core processors

Most distributed source coding schemes involve the application of a channel code to the signal and transmission of the resulting syndromes. For low-complexity encoding with superior compression performance, graph-based channel codes such as LDPC codes are used to generate the syndromes. The encoder performs simple XOR operations, while the decoder uses belief propagation (BP) decoding to recover the signal of interest using the syndromes and some correlated side information. We consider parallelization of BP decoding on general-purpose multi-core CPUs. The motivation is to make BP decoding fast enough for realtime applications. We consider three different BP decoding algorithms: Sum-Product BP, Min-Sum BP and Algorithm E. The speedup obtained by parallelizing these algorithms is examined along with the tradeoff against decoding performance. Parallelization is achieved by dividing the received syndrome vectors among different cores, and by using vector operations to simultaneously process multiple check nodes in each core. While Min-Sum BP has intermediate decoding complexity, a “vectorized” version of Min-Sum BP performs nearly as fast as the much simpler Algorithm E with significantly fewer decoding errors. Our experiments indicate that, for the best compromise between speed and performance, the decoder should use Min-Sum BP when the side information is of good quality and Sum-Product BP otherwise.
A multi-standards HDTV video decoder for blu-ray disc standard

This paper presents an HDTV video decoder core that is able to decode MPEG-2, MPEG-4 AVC and VC-1 formats and is fully compatible with the Blu-ray Disc standard. The core has two major features which achieve low-cost hardware implementation for three different standards. First, a novel re-configurable architecture is adopted to realize reduced hardware for three different variable length coding tables. A hybrid architecture that consists of a cell array and coefficients memory is introduced. By considering the trade-off between performance and cost, the cell array can compare a variable number of bits in a flexible manner. Second, a data compression method suitable for all video decoding standards is applied to reduce memory data usage and access bandwidth. A compression syntax based on Run-Level encoding with Exp-Golomb tables is used to achieve both low compression workload and reasonable memory cost. The core is implemented by top-down HDL basis approach, and the circuit volume is 1.5 Mgates with 90 nm CMOS technology and operation clock frequency is 162 MHz for 1080 i at 30 fr/s.
An online tensor robust PCA algorithm for sequential 2D data

Tensor robust principal component analysis (PCA) approaches have drawn considerable interests in many applications such as background subtraction, denoising, and outlier detection, etc. In this paper we propose an online tensor robust PCA where the multidimensional data (tensor) is revealed sequentially in online mode, and tensor PCA is updated based on the latest estimation and the newly collected data. Compared to the tensor robust PCA in batch mode, we significantly reduce the required memory and improve the computation efficiency. Application on fusing cloud-contaminated satellite images demonstrates that the proposed method shows superiority in both convergence speed and performance compared to the state-of-the-art approaches.
Privacy and security of features extracted from minutiae aggregates

This paper describes our recent analysis on the security and privacy of biometric feature vectors obtained from fingerprint minutiae. A large number of contiguous regions (cuboids) are selected at random in the minutiae space, and several new features are extracted from the minutiae inside each such cuboid. Specifically, the features are extracted from the average minutia coordinate within a cuboid, the standard deviation of the minutiae coordinates, and the aggregate wall distance, i.e., the sum of distance of each minutia from the boundary of the cuboids. In terms of matching performance on a public database, the feature vectors provide an equal error rate of 3 % even if the imposter is allowed to use the same local patches as the genuine user. Performance within a secure biometrics framework is evaluated by applying an LDPC code to the feature vectors and storing only the syndrome at the access control device, for use in authentication. The paper concludes with a discussion on methods to analyze security and privacy of biometric systems that use such local-aggregate-based feature vectors in a secure biometric recognition framework. This discussion highlights security attacks via template injection, spoofing, and cancelability compromises and also considers the difficulty of privacy attacks via template inversion.
A Study of MPEG-2 to H.264/AVC Transcoding with Half-Horizontal Resolution

This paper analyzes the performance of MPEG-2 to H.264/AVC transcoding to half-horizontal resolution. Both the quality of the resulting video and complexity of the system are examined in the context of next-generation HDD recording systems. Experimental results show the importance of motion mapping to maintain high picture quality and that quality improvement over full-resolution transcoding is possible. It is also shown that transcoding to half-horizontal resolution allows for substantial reduction in complexity.
Energy efficient JPEG 2000 image transmission over wireless sensor networks

We propose an energy efficient JPEG 2000 image transmission system over point-to-point wireless sensor networks. The objective is to minimize the overall processing and transmission energy consumption with the expected end-to-end QoS guarantee, which is achieved by jointly adjusting the source coding schemes, channel coding rates, and transmitter power levels in an optimal way. The advantages of the proposed system lie in three aspects: adaptivity, optimality, and low complexity. Based on the characteristics of the image content, the estimated channel conditions, and the distortion constraint, the proposed low-complexity joint source channel coding and power control algorithm adjusts the coding and transmission strategies adaptively, which can approximate the optimal solution with a tight bound.
3D in the Home: Mass Market or Niche?

There is substantial momentum to increase the production of digital 3D content and the number of 3D digital cinema screens. In addition to the novelty of now being able to offer an immersive high-quality multimedia experience to movie-goers, reports indicate that the average revenue per 3D screen is approximately three times higher than that of conventional 2D screens. But how does all of this translate to the emerging 3D business in the home? This talk will highlight some of the barriers involved in bringing a high-quality 3D experience to the home, and consider whether 3D could achieve mass market deployment and appeal in that space. The current state of technology and existing consumer product offerings will first be introduced. Then, the requirements for a common 3D data format will be discussed with particular emphasis on ongoing standardization efforts and related technology for multiview video coding and coded depth representations. While getting 3D content into the home and having a means to decode and display it are necessary, it is also critical to ensure a pleasant 3D viewing experience. Repurposing of 3D digital cinemas content for the home will be touch on, and some related display processing techniques for both stereoscopic and auto-multiscopic displays will also be presented. This talk will conclude with an assessment of future needs going forward.
Use of two-dimensional deformable mesh structures for video coding. II. The analysis problem and a region-based coder employing an active mesh representation

For pt.I see ibid., vol.6, no.6, p.636-46 (1996). This paper explores the use of the deformable mesh structure for motion/shape analysis and synthesis in an image sequence. We present algorithms for the analysis problem, including scene-adaptive mesh generation and node tracking over successive frames. We also describe a region-based video coder that integrates the analysis and synthesis algorithms presented. The coder describes each region by an ensemble of connected quadrilateral elements embedded in a mesh structure. For each region, its shape and texture are described by the nodal positions and image functions of the elements in this region in an initial frame, while its motion (including shape deformation) is characterized by the nodal trajectories in the following frames, which are in turn specified by a few motion parameters. This coder has been applied to a typical common intermediate format (CIF) resolution, head-and-shoulder type sequence. The visual quality is significantly better than the H.263-TMN4 algorithm at about 50 kb/s (for the luminance component only, 30 Hz).
Leveraging reliable bits: ECC design considerations for practical secure biometric systems

It is well-known that a biometric fuzzy vault can be constructed by applying an error correcting code (ECC) to a biometric signal. This is attractive because authentication only requires the check bits of the ECC to be stored on the access control device, whereas the personal biometric traits need not be stored. For a given coding rate, the ECC attempts to correct the errors between an enrollment biometric and the provided probe, and authenticates if it is successful in doing so. Unfortunately, most implementations of biometric fuzzy vaults have very poor robustness to the inherent noisiness of biometric measurements. In this paper, we provide ECC design considerations for secure biometric systems, which provide both better robustness and greater security. In particular, for any feature extraction algorithm, we propose to reorder the feature bits according to their reliability, and associate the reliable bits with high-degree variable nodes in the graph of the ECC. Further, the reliability of a bit is measured at enrollment and used to initialize the ECC decoding. Experiments on an extensive database show considerable reduction in the false reject rate, while restricting the successful attack rate to a very low value.
Depth Coding Using a Boundary Reconstruction Filter for 3-D Video Systems

A depth image is 3-D information used for virtual view synthesis in 3-D video system. In depth coding, the object boundaries are hard to compress and severely affect the rendering quality since they are sensitive to coding errors. In this paper, we propose a depth boundary reconstruction filter and utilize it as an in-loop filter to code the depth video. The proposed depth boundary reconstruction filter is designed considering occurrence frequency, similarity, and closeness of pixels. Experimental results demonstrate that the proposed depth boundary reconstruction filter is useful for efficient depth coding as well as high-quality 3-D rendering.