nils-herrmann/clean_scirepeval huggingface dataset:
Have we underestimated the power of image undistortion?

An indispensable pre-processing step for image-based 3D reconstruction or photogrammetry tasks is undistortion, which is employed for correcting the non-linear projection of the surface points of objects onto the image plane due to lens distortion. To deal with this undesirable artifact, state-of-the-art methods focus on directly recovering the 2D coordinates of image points from distorted images, where a mapping function rather than the real undistortion model is obtained. Contrary to the standard undistortion, we explicitly consider the imaging process of a finite projective camera: the projection of a 3D point onto the image plane is determined not only by the lens, but also by the intrinsic parameters of the camera. After an appropriate transformation of the intrinsic matrix for parameter reduction, it is shown that an exact recovery of the real undistorted coordinates can be achieved for most of the commonly used undistortion functions. Interestingly, a simultaneous estimation of the two intrinsic parameters, aspect ratio a and skew factor s, is also embedded in the undistortion process, which demonstrates the underestimated power of image undistortion in a conventional way. As a benefit of considering the physical model instead of fitting a mapping function, improved undistortion performance can be achieved.
Gaze latent support vector machine for image classification

This paper deals with image categorization from weak supervision, e.g. global image labels. We propose to improve the region selection performed in latent variable models such as Latent Support Vector Machine (LSVM) by leveraging human eye movement features collected from an eye-tracker device. We introduce a new model, Gaze Latent Support Vector Machine (G-LSVM), whose region selection during training is biased toward regions with a large gaze density ratio. On this purpose, the training objective is enriched with a gaze loss, from which we derive a convex upper bound, leading to a Concave-Convex Procedure (CCCP) optimization scheme. Experiments show that G-LSVM significantly outperforms LSVM in both object detection and action recognition problems on PASCAL VOC 2012. We also show that our G-LSVM is even slightly better than a model trained from bounding box annotations, while gaze labels are much cheaper to collect.
Pyramidal image representation with deformation: Reformulation of domain transform and filter designs

Edge-preserving smoothing is one of the most important topics for image and video processing. Recently, we presented a multiscale image decomposition method based on domain transform, which is an efficient edge-preserving smoothing method. It is robust to noise compared with the original domain transform. In this paper, we generalize the scheme so that it can be applied to not only domain transform but also any deformation (warping) of pixels (or nonuniformly sampled signals) through showing the facts that the domain transform is a specific method of mesh deformation with a simple cost function. We also propose construction methods of filters by using arbitrary continuous filters or graph spectral filters in order that various filters can be accommodated by our method. To evaluate the proposed method, we performed experiments on pencil drawing and stylization. These results show that our approach easily changes the smoothing strength in accordance with the applications due to its multiscale scheme.
New approach for human detection in spherical images

Omnidirectional cameras are commonly used in computer vision and robotics. Their main advantage is their wide field of view which allows them to acquire a 360 degree view of the scene with only one sensor and a single shot. However, few studies have investigated the human detection problem using this kind of cameras. In this paper, we propose to extend the conventional approach for human detection in perspective images and based on Histogram of Oriented Gradients (HOG) apdapted to spherical images is used for this issue. Our approach uses the Riemannian manifolds in order to adapt the gradient in the omnidirectional images. Several experiments have been done using INRIA image database; the results show that adapting detection and image database to the geometry of omnidirectional camera allows a robust detection, and significantly increases the performances.
Fast multidimensional image processing with OpenCL

Multidimensional image data, i.e., images with three or more dimensions, are used in many areas of science. Multidimensional image proçessing is supported in Python and MATLAB. VisionGL is an open source library that provides a set of image processing functions and can help the programmer by automatically generating code. The objective of this work is to augment VisionGL by adding multidimensional image processing support with OpenCL for high performance through use of GPUs. Benchmarking experiments were run with window and point operations to compare Python, MATLAB and VisionGL when processing 1D to 5D images. As a result, speedups of up to two orders of magnitude were obtained.
Oblique random forest based on partial least squares applied to pedestrian detection

The increasing popularity of approaches based on random forest in computer vision tasks is due to its simplicity and flexibility with complex data. Random forest is a set of decision trees that can be divided in two subsets according to the view of the feature descriptors provided as input: orthogonal and oblique. In the former, the feature space is separated orthogonally (axis-aligned) by a single feature at a time. In the latter, it separates the space by oriented hyperplanes, which usually provides better data modeling. This work proposes a novel oblique random forest associated with Partial Least Squares to perform the oblique split. We validate the proposed approach, referred to as oRF-PLS, on the challenge INRIA Person dataset. Experimental results demonstrate that the proposed method outperforms traditional state-of-the-art detectors. In addition, we demonstrate that PLS is a more suitable choice to build oblique random forest than SVM, being faster and producing more accurate forests.
Part-based multi-graph ranking for visual tracking

Recently, graph ranking-based methods have been introduced to visual tracking and achieved promising results due to the local structure preserving property. However, existing graph ranking-based trackers use holistic templates to construct the graphs which makes the trackers sensitive to occlusions. In this paper, we propose a part-based multi-graph ranking algorithm for robust visual tracking. In our method, template samples are divided into local parts. Multiple graphs are constructed based on different part samples and different feature representations. Then, the multiple graphs are integrated into a regularization framework with each graph assigned a weight. Furthermore, by imposing the l2,1 norm on the weight matrix of graphs, the confident parts are selected to reduce the effects of occluded ones. An effective optimization scheme is proposed to learn the weight matrix and the rank scores jointly. Experimental results on various challenging video sequences demonstrate our proposed algorithm outperforms state-of-the-art trackers.
Diversity promoting online sampling for streaming video summarization

Many applications benefit from sampling algorithms where a small number of well chosen samples are used to generalize different properties of a large dataset. In this paper, we use diverse sampling for streaming video summarization. Several emerging applications support streaming video, but existing summarization algorithms need access to the entire video which requires a lot of memory and computational power. We propose a memory efficient and computationally fast, online algorithm that uses competitive learning for diverse sampling. Our algorithm is a generalization of online K-means such that the cost function reduces clustering error, while also ensuring a diverse set of samples. The diversity is measured as the volume of a convex hull around the samples. Finally, the performance of the proposed algorithm is measured against human users for 50 videos in the VSUMM dataset. The algorithm performs better than batch mode summarization, while requiring significantly lower memory and computational requirements.
Webpage saliency prediction with multi-features fusion

We proposed a novel model to predict human's visual attention when free-viewing webpages. Compared with natural images, webpages are usually full of salient regions such as logos, text, and faces, while few of them attract human's attention in a short sight. Moreover, webpages perform distinct viewing patterns which are quite different from the natural images. In this paper, we introduced multi-features according to our observation on webpages characters and related eye-tracking data. Further, in order to achieve a flexible adaptation to various types of webpages, we employed a machine-learning framework based on our proposed features. Experimental results demonstrate that our model outperforms other state-of-the-art methods in webpage saliency prediction.
ML estimation of the resampling factor

In this work, the problem of resampling factor estimation for tampering detection is addressed following the maximum likelihood criterion. By relying on the rounding operation applied after resampling, an approximation of the likelihood function of the quantized resampled signal is obtained. From the underlying statistical model, the maximum likelihood estimate is derived for one-dimensional signals and a piecewise linear interpolation. The performance of the obtained estimator is evaluated, showing that it outperforms state-of-the-art methods.
An SVD approach to forensic image resampling detection

This paper describes a new strategy for image resampling detection whenever the applied resampling factor is larger than one. Delving into the linear dependencies induced in an image after the application of an upsampling operation, we show that interpolated images belong to a subspace defined by the interpolation kernel. Within this framework, by computing the SVD of a given image block and a measure of its degree of saturated pixels per row/column, we derive a simple detector capable of discriminating between upsampled images and genuine images. Furthermore, the proposed detector shows remarkable results with blocks of small size and outperforms state-of-the-art methods.
Dither-modulation data hiding with distortion-compensation: exact performance analysis and an improved detector for JPEG attacks

The binary distortion compensated dither-modulation (DC-DM), which can be regarded to as a baseline for quantization-based data-hiding methods, is rigorously analyzed. A novel and accurate procedure for computing the exact probability of bit error is given, as well as an approximation amenable to differentiation which allows to obtain the optimal weights in a newly proposed decoding structure, for significant improvements on performance. The results are particularized for a JPEG compression scenario which allows to show their usefulness. Experimental results validating the proposed theory are presented.
Introduction of New Associate Editors

I would like to welcome 22 new outstanding researchers who are joining the IEEE Transactions on Circuits and Systems for Video Technology (TCSVT) Editorial Board (EB) as Associate Editors. They are all respected and well-known experts in TCSVT-related fields. I am looking forward to working closely with them and all other existing TCSVT EB members to push TCSVT to a new level. Their profiles are as follows. Welcome aboard!
Performance Analysis of Fridrich–Goljan Self-Embedding Authentication Method

This paper analyzes the performance of the image authentication method based on robust hashing proposed by J. Fridrich and M. Goljan . In this method, both the embedder and the detector generate the watermark from a perceptual digest of the image. Therefore, an accurate performance analysis requires assessing the relation between noise and hash bit errors. Our approach first derives the probability of hash bit error due to watermark embedding and/or the attack, and then uses such probability to derive the probabilities of false positive and false negative.
Smart Detection of Line-Search Oracle Attacks

We discuss how a binary detector can learn whether it is being subject to an oracle attack by resorting to a higher level of detection (metadetection). On a second step, assuming that the attacker is aware of the fact that the detector takes countermeasures, we investigate a possible way for him to react. Then, we study the interplay between the defender and the attacker when both of them try to do their best for pursuing their opposite goals. We focus our analysis on the metadetection of oracle attacks based on line search algorithms, as they are prevalent in the literature. In such scenario, we propose metadetectors, which work under very general conditions, that is, when the oracle is not exclusively fed with line search attacking queries, but only some of the malicious queries are made along the lines, whereas the others are done by mimicking the behavior of honest users. We theoretically evaluate the final achievable performance of these metadetectors, deriving conditions under which asymptotic powerful testing is possible. Experimental results show the power of metadetection for countering the line search attacks in both synthetic and real application scenarios.
Quantization lattice estimation for multimedia forensics

The most widely used information lossy source coding schemes for multimedia signals rely on the quantization of the content samples in a linearly transformed domain. A number of forensic applications (e.g., processing history estimation, tampering detection, software identification) can be posed as the estimation of the equivalent lattice quantizer from the observed samples. We present a new lattice estimation algorithm based on the observation of noisy points of the lattice. Although inspired by the work of Neelamani et al., our scheme uses the so-called “dual lattice” to achieve significant performance improvements with respect to its predecessors as measured by the number of vectors of the lattice basis that can be correctly estimated. Such performance improvement is even more dramatic when small pieces of the contents are considered, which indeed is especially relevant for forensic applications.
On a Watermarking Scheme in the Logarithmic Domain and its Perceptual Advantages

Scaling attacks are well-known to be some of the most harmful strategies against quantization-based watermarking methods, as they can completely ruin the performance of the watermarking system with almost no perceptual impact on the watermarked signal. In this paper we propose a new family of quantization-based methods specifically devised to deal with those attacks, and which presents the desirable property of yielding perceptually shaped watermarks.
On the effectiveness of meta-detection for countering oracle attacks in watermarking

We evaluate the performance of smart metadetection as a way to combat oracle attacks in watermarking. In a recent work, we have shown that few queries are sufficient for a simple metadetector (namely, a metadetector based on the closeness of queries to the watermark detection boundary) to detect an oracle attack. A limitation of our prior analysis is the assumption that all the queries correspond to either honest users or malicious ones. In this paper, we address a more realistic scenario in which honest queries are interspersed with queries derived from an oracle attack. By focusing on this more general situation, we evaluate the performance of the metadetection and derive conditions under which powerful testing is possible.
Taking advantage of source correlation in forensic analysis

In a wide range of practical multimedia scenarios several correlated contents are available. The aim of this work is to quantify the gain that can be achieved in forensic applications by jointly considering those contents, instead of analyzing them separately. The used tool is the Kullback-Leibler Divergence between the distributions corresponding to different operators; the Maximum Likelihood estimator of the applied operator is also obtained, in order to illustrate how the correlation is exploited for estimation. Our detailed analysis is constrained to the Gaussian case (both for the input signal distribution and the processing randomness) and linear operators. Several practical scenarios are studied, and the relationships between the derived results are established. Finally, the links with Distributed Source Coding are highlighted.
Asymptotically Optimum Universal Watermark Embedding and Detection in the High-SNR Regime

The problem of optimum watermark embedding and detection was addressed in a recent paper by Merhav and Sabbag, where the optimality criterion was the maximum false-negative error exponent subject to a guaranteed false-positive error exponent. In particular, Merhav and Sabbag derived universal asymptotically optimum embedding and detection rules under the assumption that the detector relies solely on second-order joint empirical statistics of the received signal and the watermark. In the case of a Gaussian host signal and a Gaussian attack, however, closed-form expressions for the optimum embedding strategy and the false-negative error exponent were not obtained in that work. In this paper, we derive the false-negative error exponent for any given embedding strategy and use such a result to show that in general the optimum embedding rule depends on the variance of the host sequence and the variance of the attack noise. We then focus on high signal-to-noise ratio (SNR) regime, deriving the optimum embedding strategy for such a setup. In this case, a universally optimum embedding rule turns out to exist and to be very simple with an intuitively appealing geometrical interpretation. The effectiveness of the newly proposed embedding strategy is evaluated numerically.
Set-membership identification of resampled signals

The problem of resampling factor estimation as a means for tampering detection has been largely investigated. Most of the existing techniques rely on the analysis of cyclic correlations induced in the resampled signal. However, in this paper, a new direction is explored by addressing the same problem in terms of the set-membership estimation theory. The proposed technique constructs a model of the problem using available a priori knowledge and in consonance with a finite number of observations that comes from the resampled signal under study. With this information, the proposed technique is able to provide an estimate of the resampling factor applied to the original signal and, if required, an estimate of such signal and an estimate of the interpolation filter. The performance in terms of accuracy and MSE of the proposed approach is evaluated and comparative results with state-of-the-art methods are reported.
Transportation-theoretic image counterforensics to First Significant Digit histogram forensics

First-order statistics of First Significant Digits (FSD) have been recently exploited in multimedia forensics as a powerful tool to reveal traces of previous coding operations. As an answer, adversarial approaches aimed at modifying the FSD histogram and fooling such forensic methods have been proposed. However, the existing techniques have limitations in terms of distortion introduced in the multimedia object. In this paper, a transportation-theoretic formulation of the problem is presented which provides a close-to-optimal solution. Such strategy is tested in a well-known image forensic scenario, where FSDs of 8 × 8-DCT coefficients after single or double quantization are modified in order to restore a certain target histogram and the distortion with respect to the provided compressed image is measured in terms of MSE.
A new look at ML step-size estimation for Scalar Costa scheme data hiding

Watermarking schemes based on the Dirty Paper Coding (DPC) paradigm have been shown to achieve much higher rates than classical Spread-Spectrum methods. However, in practice, the latter continue to be used due to their higher security and robustness. In fact, the most prevalent DPC method, the so-called Scalar Costa Scheme (SCS), is prone to non-additive attacks, such as a simple gain which produces a desynchronization between the embedding and decoding codebooks thus severely affecting performance. Although some gain-robust modifications to the basic SCS exist, all have serious drawbacks. One alternative, which was somehow abandoned for its complexity, is to estimate the gain at the decoder, with the advantage of preserving the simplicity of SCS. In this paper we take a new look at the estimation problem and propose an affordable algorithm to perform Maximum Likelihood estimation of the channel gain, that is able to restore the original SCS performance. We also show and experimentally illustrate how our scheme can be effectively adapted to watermark decoding in filtered images.
Flat fading channel estimation based on Dirty Paper Coding

A novel complex flat fading channel estimation scheme is proposed. Contrarily to previous schemes in the literature, this new approach is not based on introducing pilot sequences, but on reducing the interference caused by the information-bearing signal on the estimation-aiding signal by using Dirty Paper Coding. We show through simulations that our method outperforms the Partially-Data Dependent scheme, which is a state-of-the-art technique based on superimposed pilots.
Perfomance analysis of the Fridrich-Goljan self-embedding authentication method

A performance analysis of the authentication method proposed by J. Fridrich and M. Goljan is carried out. This method has the particular feature that both the embedder and the detector generate the watermark from a perceptual digest of the image. Hence, in order to accurately analyze the performance, the digest errors caused by the watermark embedding, the addition of a complementary signal and the scaling attacks are also taken into account.
On distortion-compensated dither modulation data-hiding with repetition coding

An exhaustive analysis of the distortion-compensated dither modulation (DC-DM) data-hiding method with repetition coding is presented. Two decoding strategies, maximum likelihood lattice decoding and Euclidean distance decoding, are discussed and some simplifications presented. An exact performance analysis in terms of the bit error rate (BER) is given; such an exact analysis is currently not available in the literature. Two methods for computing the exact BER and several approximations and bounds, most of them in closed form, are provided. These approximations are employed to propose two novel improvements on the standard DC-DM method with repetition: the use of a weighted Euclidean distance, with optimizable weights, and a vector form of the distortion compensation parameter. Both account for significant performance improvements. DC-DM is compared with quantization methods in the projected domain, showing worse performance against additive noise attacks but higher robustness to cropping attacks. A performance analysis of DC-DM under coarse quantization that can be specialized to JPEG compression is also supplied. All our results are validated with numerical simulations with both synthetic data and real images.