nils-herrmann/clean_scirepeval huggingface dataset:
Subjective study of privacy filters in video surveillance

Extensive adoption of video surveillance, affecting many aspects of the daily life, alarms the concerned public about the increasing invasion into personal privacy. Therefore, to address privacy issues, many tools have been proposed for protection of personal privacy in image and video. However, little is understood regarding the effectiveness of such tools and especially their impact on the underlying surveillance tasks. In this paper, we propose a subjective evaluation methodology to analyze the tradeoff between the preservation of privacy offered by these tools and the intelligibility of activities under video surveillance. As an example, the proposed method is used to compare several commonly employed privacy protection techniques, such as blurring, pixelization, and masking applied to indoor surveillance video. The results show that, for the test material under analysis, the pixelization filter provides the best performance in terms of balance between privacy protection and intelligibility.
Still-image watermarking robust to local geometric distortions

Geometrical distortions are the Achilles heel for many watermarking schemes. Most countermeasures proposed in the literature only address the problem of global affine transforms (e.g., rotation, scaling, and translation). In this paper, we propose an original blind watermarking algorithm robust to local geometrical distortions such as the deformations induced by Stirmark. Our method consists in adding a predefined additional information to the useful message bits at the insertion step. These additional bits are labeled as resynchronization bits or reference bits and they are modulated in the same way as the information bits. During the extraction step, the reference bits are used as anchor points to estimate and compensate for small local and global geometrical distortions. The deformations are approximated using a modified basic optical flow algorithm
Reversible watermarking of 3D mesh models by prediction-error expansion

In this paper, a reversible watermarking algorithm is proposed for 3D mesh models based on prediction-error expansion. Firstly, we predict a vertex position by calculating the centroid of its traversed neighbors. Then the prediction error, i.e. the difference between the predicted and real positions, is expanded for data embedding. So only the vertex coordinates are modified to embed a watermark into the mesh content without changing the topology. We further reduce the distortion by adaptively choosing a threshold so that the prediction errors with too large magnitude will not be expanded. The chosen threshold value and critical location information should be saved in the watermarked mesh to guide the recovery process. The experiments show that the original mesh can be exactly recovered and consequently our algorithm can be used for symmetric or public key authentication of 3D mesh models.
Toward generic image dewatermarking?

During the last decade, a significant effort has been put into designing watermarking algorithms. The watermarking community now needs some advanced attacks and fair benchmarks in order to compare the performances of different watermarking technologies. Moreover, attacks permit the weaknesses of an algorithm to be found and consequently trigger further research in order to overcome the problem. These ideas motivated the creation of the European Certimark project. After a short definition of dewatermarking, we present an original attack based on self similarities. This attack is then put to the test with three different publicly available watermarking tools. Finally, we discuss briefly the feasibility of a generic attack i.e. a dewatermarking attack which should succeed in removing whatever watermarks have been inserted by whatever watermarking tools.
Crowd density map estimation based on feature tracks

Crowd density analysis is a crucial component in visual surveillance mainly for security monitoring. This paper proposes a novel approach for crowd density measure, in which local information at pixel level substitutes a global crowd level or a number of people per-frame. The proposed approach consists of generating fully automatic and crowd density maps using local features as an observation of a probabilistic crowd function. It also involves a feature tracking step which allows excluding feature points belonging to the background. This process is favorable for the later density function estimation since the influence of features irrelevant to the underlying crowd density is removed. Our proposed approach is evaluated on videos from different datasets, and the results demonstrate the effectiveness of feature tracks for crowd estimation. Furthermore, we include a comparative study between different local features in order to investigate their discriminative power to the crowd.
A novel indexing approach for multimedia image databases

This paper proposes an algorithm for content-based image indexing whose formulation of similarity is borrowed from methods for fractal compression. Unlike most traditional block-based image indexing algorithms, the proposed method employs dynamic programming to exploit inter-block dependencies. A regularization constraint is globally imposed, and the overall optimal transformation relating two images is efficiently determined by application of the Viterbi algorithm. Preliminary experimental results on a sample of eighty binary images from the MPEG-7 database are presented.
Face aging with conditional generative adversarial networks

It has been recently shown that Generative Adversarial Networks (GANs) can produce synthetic images of exceptional visual fidelity. In this work, we propose the first GAN-based method for automatic face aging. Contrary to previous works employing GANs for altering of facial attributes, we make a particular emphasize on preserving the original person's identity in the aged version of his/her face. To this end, we introduce a novel approach for “Identity-Preserving” optimization of GAN's latent vectors. The objective evaluation of the resulting aged and rejuvenated face images by the state-of-the-art face recognition and age estimation solutions demonstrate the high potential of the proposed method.
Low level crowd analysis using frame-wise normalized feature for people counting

People counting is a crucial component in visual surveillance mainly for crowd monitoring and management. Recently, significant progress has been made in this field by using features regression. In this context, perspective distortions have been frequently studied, however, crowded scenes remain particularly challenging and could deeply affect the count because of the partial occlusions that occur between individuals. To address these challenges, we propose a people counting approach that harness the advantage of incorporating an uniform motion model into Gaussian Mixture Model (GMM) background subtraction to obtain high accurate foreground segmentation. The counting is based on foreground measurements, where a perspective normalization and a crowd measure-informed corner density are introduced with foreground pixel counts into a single feature. Afterwards, the correspondence between this frame-wise feature and the number of persons is learned by Gaussian Process regression. Experimental results demonstrate the benefits of integrating GMM with motion cue, and normalizing the proposed feature as well. Also, by means of comparisons to other feature-based methods, our approach has been experimentally validated showing more accurate results.
Facial gender recognition using multiple sources of visual information

In this article we present a novel multimodal gender recognition system, which successfully integrates the head and mouth motion information with facial appearance by taking advantage of a unified probabilistic framework. In fact, we develop a temporal subsystem that has an extended feature space consisting of parameters related to head and mouth motion; at the same time, we introduce a complementary spatial subsystem based on a probabilistic extension of the eigenface approach. In the end, we implement an integration step to combine the similarity scores of the two parallel subsystems, using a suitable opinion fusion (or score fusion) strategy. The experiments show that not only facial appearance but also head and mouth motion possess a potentially relevant discriminatory power, and that the integration of different sources of biometric information from video sequences is the key strategy to develop more accurate and reliable recognition systems.
Synthetic and privacy-preserving visualization of video sensor network outputs

We propose a complete framework for people tracking in video surveillance networks that preserves privacy by mapping people recorded by a video sensor network in the map of the corresponding surveilled areas. Thanks to this approach, it is possible (1) to synthesize multiple video outputs into a unique picture, and (2) to control privacy by offering the possibility to filter information to viewers. A set of physical attributes, namely soft biometric characteristics, is used to provide the system with tracking capabilities. We demonstrate the feasibility of our approach by showing a real case application scenario.
CO-LDA: A Semi-supervised Approach to Audio-Visual Person Recognition

Client models used in Automatic Speaker Recognition (ASR) and Automatic Face Recognition (AFR) are usually trained with labelled data acquired in a small number of menthol sessions. The amount of training data is rarely sufficient to reliably represent the variation which occurs later during testing. Larger quantities of client-specific training data can always be obtained, but manual collection and labelling is often cost-prohibitive. Co-training, a paradigm of semi-supervised machine learning, which can exploit unlabelled data to enhance weakly learned client models. In this paper, we propose a co-LDA algorithm which uses both labelled and unlabelled data to capture greater intersession variation and to learn discriminative subspaces in which test examples can be more accurately classified. The proposed algorithm is naturally suited to audio-visual person recognition because vocal and visual biometric features intrinsically satisfy the assumptions of feature sufficiency and independency which guarantee the effectiveness of co-training. When tested on the MOBIO database, the proposed co-training system raises a baseline identification rate from 71% to 99% while in a verification task the Equal Error Rate (EER) is reduced from 18% to about 1%. To our knowledge, this is the first successful application of co-training in audio-visual biometric systems.
Iterative decoding of two-dimensional hidden Markov models

While the hidden Markov model (HMM) has been extensively applied to one-dimensional problems, the complexity of its extension to two-dimensions grows exponentially with the data size and is intractable in most cases of interest. We introduce an efficient algorithm for approximate decoding of 2D HMMs, i.e., searching for the most likely state sequence. The basic idea is to approximate a 2D HMM with a turbo-HMM (T-HMM), which consists of horizontal and vertical 1D HMMs that "communicate", and allow iterated decoding (ID) of rows and columns by a modified version of the forward-backward algorithm. We derive the approach and its re-estimation equations. We then compare its performance to another algorithm designed for decoding 2D HMMs: the path constrained variable state Viterbi (PCVSV) algorithm (Li, J. et al.,  2000). Finally, we combine our approach with PCVSV and show that the combination outperforms each algorithm taken separately.
Countermeasure for the protection of face recognition systems against mask attacks

There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique is proposed to protect face recognition systems against mask attacks. To the best of our knowledge, this is the first time a countermeasure is proposed to detect mask attacks. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, a 2D+3D face mask attacks database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated on both the texture images and the depth maps, separately. The results show that the proposed countermeasure gives satisfactory results using both the texture images and the depth maps. The performance of the countermeasure is observed to be slight better when the technique is applied on texture images instead of depth maps, which proves that face texture provides more information than 3D face shape characteristics using the proposed approach.
Obtaining MPEG-4 compliant animatable 3D face models by using TPS method

3D surface data which yields siginificant improvements in face recognition against pose and illumination variations, is insufficient by itself in the presence of facial expressions. The proposed methods in this scope, generally choose to eliminate or avoid the adverse effect of the facial expressions by analyzing only the expression-proof regions or weighing facial regions according to their robustness. In order to overcome facial expressions issue for face recognition, the facial expressions for each object can be proposed to be obtained and learned. In this paper, we present an approach to obtain dynamic models on which the facial expressions can be animated from the static ones by utilizing the TPS (Thin Plate Spline) method. Initially, 3D frontal and neutral face models of the subjects and some of the MPEG-4 specified feature points on that models are assumed to be available. Afterwards, an animatable generic model is warped by using those models and points and hence animatable models for the subjects are obtained.
Reflectance analysis based countermeasure technique to detect face mask attacks

Face photographs, videos or masks can be used to spoof face recognition systems. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique, which analyzes the reflectance characteristics of masks and real faces, is proposed to detect mask attacks. There are limited studies on countermeasures against mask attacks. The reason for this delay is mainly due to the unavailability of public mask attack databases. In this study, a 2D+3D face mask attack database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated using the texture images which were captured during the acquisition of 3D scans. The results of the proposed countermeasure outperform the results of existing techniques, achieving a classification accuracy of 94.47%. In this paper, it is also proved that reflectance analysis may provide more information for the purpose of mask spoofing detection compared to texture analysis.
Facial cosmetics database and impact analysis on automatic face recognition

Facial cosmetics, also called makeup, may change the appearance of a face that we could perceive. In order to contribute to studies in image processing related to facial cosmetics, a database is built which contains multiple images per person with and without applied cosmetics. Annotations provide detailed information about the amount and location of applied makeup for each picture. Furthermore, a classification approach is presented. The classification approach takes the altering effect of the applied cosmetics and the application area into account. Since automatic face recognition evolved to an important topic over the last decades and is affected by facial cosmetics, preliminary tests are done to evaluate their impact on automatic face recognition. The face as a whole as well as its most significant makeup application areas that are skin, eyes and mouth are investigated separately.
ASePPI: Robust Privacy Protection Against De-Anonymization Attacks

The evolution of the video surveillance systems generates questions concerning protection of individual privacy. In this paper, we design ASePPI, an Adaptive Scrambling enabling Privacy Protection and Intelligibility method operating in the H.264/AVC stream with the aim to be robust against de-anonymization attacks targeting the restoration of the original image and the re-identification of people. The proposed approach automatically adapts the level of protection according to the resolution of the region of interest. Compared to existing methods, our framework provides a better trade-off between the privacy protection and the visibility of the scene with robustness against de-anonymization attacks. Moreover, the impact on the source coding stream is negligible.
Search pruning in video surveillance systems: Efficiency-reliability tradeoff

In the setting of computer vision, algorithmic searches often aim to identify an object of interest inside large sets of images or videos. Towards reducing the often astronomical complexity of this search, one can use pruning to filter out objects that are sufficiently distinct from the object of interest, thus resulting in a pruning gain of an overall reduced search space. Motivated by practical computer vision based scenarios such as time-constrained human identification in biometric-based video surveillance systems, we analyze the stochastic behavior of time-restricted search pruning, over large and unstructured data sets which are furthermore random and varying, and where in addition, pruning itself is not fully reliable but is instead prone to errors. In this stochastic setting we apply the information theoretic method of types as well as information divergence techniques to explore the natural tradeoff that appears between pruning gain and reliability, and proceed to study the typical and atypical gainreliability behavior, giving insight on how often pruning might fail to substantially reduce the search space. The result, as is, applies to a plethora of computer vision based applications where efficiency and reliability are intertwined bottlenecks in the overall system performance, and the simplicity of the obtained expressions allows for rigorous and insightful assessment of the pruning gain-reliability behavior in such applications, as well as for intuition into designing general object recognition systems.
A coding scheme for stereoscopic television sequences based on motion estimation-compensation using a 3D approach

The implementation of a new television service (in this case stereoscopic television or 3DTV) requires research into four fundamental areas: pick-up equipment, transmission (including, bandwidth reduction), display and picture quality. This paper focuses on the algorithm aspects for the definition of an optimal coder well-suited to 3DTV. We present a 3D approach linking 2D left and right motion in order to define a 3D coding scheme based on motion estimation-compensation. This 3D scheme is more efficient than current 2D schemes because it allows for the possibility of using a unique channel for transmitting motion estimation-compensation information instead of two or three channels motion (2D left motion, 2D right motion and eventually 2D disparity). A 3D approach could then offer new possibilities for increasing the compression rate (by about a factor 2 for additional motion information) by merging all 2D information in an unique set of 3D information.
Crowd context-dependent privacy protection filters

While various privacy protection filters have been proposed in the literature, little importance has been given to the context relevance of these filters. In this paper, we specifically focus on the dependency between privacy preservation and crowd density. We show that information about the crowd density in a scene can be used in order to adjust the level of privacy protection according to the local needs. For the estimation of density maps, we use an approach based on FAST feature extraction and local optical flow computation which allow excluding feature points on the background. This process is favorable for the later density function estimation since the influence of features irrelevant to the crowd density is removed. Afterwards, we adapt the protection level of personal privacy in videos according to the crowd density. The effectiveness of the proposed framework is evaluated with videos from different crowd datasets.
Blind detection of malicious alterations on still images using robust watermarks

Digital image manipulation software is now readily available on personal computers. It is therefore very simple to tamper with any image and make it available to others. Insuring digital image integrity becomes a major issue. In this paper, we propose an original method to protect image authenticity using an invisible and robust watermark. Our scheme is independent of the signer, however the latter must have a high capacity and be able to extract the watermark in full blind detection mode. Our approach is based on the extraction of features from the image. These features are chosen so as to be unaffected by non malicious alterations such as lossy compression. They are embedded in the image using an iterative process so that watermarked image features and information contained in the watermark coincide perfectly. The authenticity is verified by comparing the features of the tested image, with those of the original image recovered from the watermark. (6 pages)
Crowd density analysis using subspace learning on local binary pattern

Crowd density analysis is a crucial component in visual surveillance for security monitoring. This paper proposes a novel approach for crowd density estimation. The main contribution of this paper is two-fold: First, we propose to estimate crowd density at patch level, where the size of each patch varies in such way to compensate the effects of perspective distortions; second, instead of using raw features to represent each patch sample, we propose to learn a discriminant subspace of the high-dimensional Local Binary Pattern (LBP) raw feature vector where samples of different crowd density are optimally separated. The effectiveness of the proposed algorithm is evaluated on PETS dataset, and the results show that effective dimensionality reduction (DR) techniques significantly enhance the classification accuracy. The performance of the proposed framework is also compared to other frequently used features in crowd density estimation. Our proposed algorithm outperforms the state-of-the-art methods with a significant margin.
Texture-based watermarking of 3D video objects

We describe a novel framework for watermarking three-dimensional (3D) video objects via their texture information. Unlike classical algorithms dealing with 3D objects that operate on meshes in order to protect the object itself, the main goal of our work is to retrieve information originally hidden in the textured image of the object from resulting images or videos having used the 3D synthetic object, thus protecting the visual representations of the object. After developing the theory and practical details of this 3D object watermarking scheme, we present the results of several experiments carried out in various conditions, ranging from ideal conditions (e.g., known rendering parameters) to more realistic conditions (e.g, unknown rendering parameters, but estimated from a two-dimensional view) or within the context of attacks (e.g., mesh reduction).