nils-herrmann/clean_scirepeval huggingface dataset:
Real-time denoising of ToF measurements by spatio-temporal non-local mean filtering

This work addresses the problem of denoising of range data obtained by ToF continuous signal modulation camera working in low-power mode. The proposed approach is based on non-local mean filtering applied over extensive spatio-temporal block search in complex-valued signal domain. The extensive search allows for using shorter integration times of the range sensors and leads to an effective overcomplete structure suitable for denoising. The filter structure is optimized for real-time operation and achieves O(1) performance for arbitrary patch size by utilizing summed area tables and look-up table data fetching. The experimental results show practically the same performance while compared with state-of-the-art approaches, for greatly improved speed.
Mobile 3D video broadcast

In this paper, we present a complete framework of an end-to-end error resilient transmission of 3D video over Digital Video Broadcasting - Handheld (DVB-H) and provide an extensive analysis of coding and transmission parameters. We perform the analysis for different coding and error resilience schemes using different contents coded at different bitrate levels. Throughout the experiments, we investigate the effects of video content type, video bitrate, coding method and unequal protection level for different channel conditions. The results show that Multi-view Coding (MVC) coding outperforms Simulcast and distribution of available bitrate between video quality and Forward Error Correction (FEC) protection is an important factor in different channel conditions.
A Survey of Signal Processing Problems and Tools in Holographic Three-Dimensional Television

Diffraction and holography are fertile areas for application of signal theory and processing. Recent work on 3DTV displays has posed particularly challenging signal processing problems. Various procedures to compute Rayleigh-Sommerfeld, Fresnel and Fraunhofer diffraction exist in the literature. Diffraction between parallel planes and tilted planes can be efficiently computed. Discretization and quantization of diffraction fields yield interesting theoretical and practical results, and allow efficient schemes compared to commonly used Nyquist sampling. The literature on computer-generated holography provides a good resource for holographic 3DTV related issues. Fast algorithms to compute Fourier, Walsh-Hadamard, fractional Fourier, linear canonical, Fresnel, and wavelet transforms, as well as optimization-based techniques such as best orthogonal basis, matching pursuit, basis pursuit etc., are especially relevant signal processing techniques for wave propagation, diffraction, holography, and related problems. Atomic decompositions, multiresolution techniques, Gabor functions, and Wigner distributions are among the signal processing techniques which have or may be applied to problems in optics. Research aimed at solving such problems at the intersection of wave optics and signal processing promises not only to facilitate the development of 3DTV systems, but also to contribute to fundamental advances in optics and signal processing theory.
Measurement of perceived spatial resolution in 3D light-field displays

Effective spatial resolution of projection-based 3D light-field (LF) displays is an important quantity, which is informative about the capabilities of the display to recreate views in space and is important for content creation. We propose a subjective experiment to measure the spatial resolution of LF displays and compare it to our objective measurement technique. The subjective experiment determines the limit of visibility on the screen as perceived by viewers. The test involves subjects determining the direction of patterns that resemble tumbling E eye test charts. These results are checked against the LF display resolution determined by objective means. The objective measurement models the display as a signal-processing channel. It characterizes the display throughput in terms of passband, quantified by spatial resolution measurements in multiple directions. We also explore the effect of viewing angle and motion parallax on the spatial resolution.
A fast image segmentation algorithm using color and depth map

In this paper, a real-time image segmentation algorithm is presented. It utilizes both color and depth information retrieved from a multi-sensor capture system, which combines stereo camera pairs with time-of-flight range sensor. The algorithm targets low complexity and fast implementation which can be achieved through parallelization. Applications, such as immersive videoconferencing and lecturer segmentation for augmented reality lecture presentation, can benefit from the designed algorithm.
Anti-aliasing filtering of 2D images for multi-view auto-stereoscopic displays

In this paper, we address the problem of anti-aliasing filtering of images to be displayed on auto-stereoscopic displays. Auto-stereoscopic displays are constructed to create 3D visual effect by no special glasses but utilizing extra optical layer to cast different images to different directions. The topology of such layer is a compromise between the number of different views generated and the spatial resolution per view being fraction of the full 2D spatial resolution. Usually, the compromise is achieved by slanted and non-rectangular sub-sampling grids causing however corresponding aliasing artefacts. These artefacts are especially visible and annoying when 2D imagery, such as graphics and text, is to be displayed on auto-stereoscopic displays. In our work, we design efficient anti-aliasing filters to mitigate this effect. Two classes of filters are studied for a 3D display case. The first class is the class of non-separable filters, which takes into account the non-rectangular topology of the particular sub-sampling grid and the effect of inter-view crosstalk, and aims at suppressing the respective aliasing replicas appearing in non-rectangular positions on the 2D Fourier plane. The second class is the class of efficient separable 2D filters based on 1D anti-aliasing filter design. We demonstrate that the latter class results in subjectively higher quality images. Studying this particular case further, we design filters for different types of imagery, distinguishing between text and graphics and also between ‘smooth’ and ‘sharp’ target anti-aliased images. As it is difficult to quantify the results by objective measures, we illustrate them by visual examples. Subjective inspections have also confirmed the feasibility of our approach.
Skin detection and tracking by monocular vision

This paper deals with area detection and tracking using color-based image processing. The application addressed concerns human head and hands tracking in real time. The authors proposed an original method in order to locate colored areas, for a foreground image. Foreground image is provided by a probabilistic way using Gaussian mixture models (GMM) of probability density functions. The temporal tracking is achieved by a particle filter, which is well adapted to partial occlusions and non Gaussian models.
A model-based approach for compound leaves understanding and identification

In this paper, we propose a specific method for the identification of compound-leaved tree species, with the aim of integrating it in an educational smartphone application. Our work is based on dedicated shape models for compound leaves, designed to estimate the number and shape of leaflets. A deformable template approach is used to fit these models and produce a high-level interpretation of the image content. The resulting models are later used for the segmentation of leaves in both plain and natural background images, by the use of multiple region-based active contours. Combined with other botany-inspired descriptors accounting for the morphological properties of the leaves, we propose a classification method that makes a semantic interpretation possible. Results are presented over a set of more than 1000 images from 17 European tree species, and an integration in the existing mobile application Folia1 is considered.
Realtime head and hands tracking by monocular vision

This paper deals with areas detection and tracking using color based image processing. The application addressed concerns human head and hands tracking. We propose an original method in order to locate, for a foreground image, colored areas. Foreground image is provided by a probabilistic way using Gaussian mixture models (GMM) of probability density functions. The temporal tracking is then achieved by a particle filter, which is well adapted to partial occlusions and non Gaussian models.
A benchmark for Background Subtraction Algorithms in monocular vision: A comparative study

Background subtraction of video sequences is mainly regarded as a solved problem. However, no complete benchmark about Background Subtraction Algorithms (BSA) has been established, with ground truth and associated quality measures. One of the reasons is that such comparative study needs annotated datasets. In this article, we propose a BSA evaluation dataset built from realistic synthetic image and we compare six BSA, according to several quality measures.
Smoothed shock filtered defuzzification with Zernike moments for liver tumor extraction in MR images

In this paper, we propose to segment liver tumor within ROI (Regions Of Interest) of MR (Magnetic Resonance) images by combining the smoothed shock noise reduction filter, that we introduced in a previous work, and a fuzzy approach. Besides the whole pipeline dedicated to our application, the originality of our method is to combine fuzzy information (membership degree to clusters) and geometrical features (Zernike moments) in the final defuzzification step. Thanks to a dataset representative of the variability of tumors' geometries and pixel intensities, we demonstrate that our approach is accurate, in every cases of liver lesions of our benchmark. We also show that the combination of both our two tools (denoising filtering and fuzzy segmentation) is the reason of the efficiency of this methodology.
Tree Leaves Extraction in Natural Images: Comparative Study of Preprocessing Tools and Segmentation Methods

In this paper, we propose a comparative study of various segmentation methods applied to the extraction of tree leaves from natural images. This study follows the design of a mobile application, developed by Cerutti et al. (published in ReVeS Participation-Tree Species Classification Using Random Forests and Botanical Features. CLEF 2012), to highlight the impact of the choices made for segmentation aspects. All the tests are based on a database of 232 images of tree leaves depicted on natural background from smartphones acquisitions. We also propose to study the improvements, in terms of performance, using preprocessing tools, such as the interaction between the user and the application through an input stroke, as well as the use of color distance maps. The results presented in this paper shows that the method developed by Cerutti et al. (denoted Guided Active Contour), obtains the best score for almost all observation criteria. Finally, we detail our online benchmark composed of 14 unsupervised methods and 6 supervised ones.
Joint hierarchical learning for efficient multi-class object detection

In addition to multi-class classification, the multi-class object detection task consists further in classifying a dominating background label. In this work, we present a novel approach where relevant classes are ranked higher and background labels are rejected. To this end, we arrange the classes into a tree structure where the classifiers are trained in a joint framework combining ranking and classification constraints. Our convex problem formulation naturally allows to apply a tree traversal algorithm that searches for the best class label and progressively rejects background labels. We evaluate our approach on the PASCAL VOC 2007 dataset and show a considerable speed-up of the detection time with increased detection performance.
Locating 3D Object Proposals: A Depth-Based Online Approach

2D object proposals, quickly detected regions in an image that likely contain an object of interest, are an effective approach for improving the computational efficiency and accuracy of object detection in color images. In this paper, we propose a novel online method that generates 3D object proposals in an RGB-D video sequence. Our main observation is that depth images provide important information about the geometry of the scene. Diverging from the traditional goal of 2D object proposals to provide a high recall, we aim for precise 3D proposals. We leverage on depth information per frame and multiview scene information to obtain accurate 3D object proposals. Using efficient but robust registration enables us to combine multiple frames of a scene in near real time and generate 3D bounding boxes for potential 3D regions of interest. Using standard metrics, such as precision-recall (P-R) curves and F-measure, we show that the proposed approach is significantly more accurate than the current state-of-the-art techniques. Our online approach can be integrated into simultaneous localization and mapping-based video processing for quick 3D object localization. Our method takes less than a second in MATLAB on the UW-RGBD scene data set on a single thread CPU and, thus, has potential to be used in low-power chips in unmanned aerial vehicles, quadcopters, and drones.
Learning deep features for multiple object tracking by using a multi-task learning strategy

Model-free object tracking is still challenging because of the limited prior knowledge and the unexpected variation of the target object. In this paper, we propose a feature learning algorithm for model-free multiple object tracking. First, we pre-learn generic features invariant to diverse motion transformations from auxiliary video data by using a deep network of anto-encoder. Then, we adapt the pre-learned features according to multiple target objects respectively in a multi-task learning manner. We treat the feature adaptation for each target object as one single task. We simultaneously learn the common feature shared by all target objects and the individual feature of each object. Experimental results demonstrate that our feature learning algorithm can significantly improve multiple object tracking performance.
On the Duality of Forward and Inverse Light Transport

Inverse light transport seeks to undo global illumination effects, such as interreflections, that pervade images of most scenes. This paper presents the theoretical and computational foundations for inverse light transport as a dual of forward rendering. Mathematically, this duality is established through the existence of underlying Neumann series expansions. Physically, it can be shown that each term of our inverse series cancels an interreflection bounce, just as the forward series adds them. While the convergence properties of the forward series are well known, we show that the oscillatory convergence of the inverse series leads to more interesting conditions on material reflectance. Conceptually, the inverse problem requires the inversion of a large light transport matrix, which is impractical for realistic resolutions using standard techniques. A natural consequence of our theoretical framework is a suite of fast computational algorithms for light transport inversion-analogous to finite element radiosity, Monte Carlo and wavelet-based methods in forward rendering-that rely at most on matrix-vector multiplications. We demonstrate two practical applications, namely, separation of individual bounces of the light transport and fast projector radiometric compensation, to display images free of global illumination artifacts in real-world environments.
A practical watermarking scheme aligned with compressed-domain CIF-to-QCIF video transcoding

Watermarking schemes are robust to lossy compression, scaling, etc. are presented in this paper. However, in real application a practical watermarking solution should still depend on the specific requirements in order to design the whole system in an effective and efficient way. In this paper, we present a compressed-domain watermarking scheme that closely aligns with MPEG transcoding from the common intermediate format (CIF) to the quarter common intermediate format (QCIF). The watermark is embedded in CIF while the watermark could be extracted from either CIF or QCIF, both in DCT domain. The simulation results demonstrate that the proposed scheme is not only robust to multicycle MPEG recoding and CIF to QCIF video transcoding, but also achieves a low computational complexity.
Comparison of human and machine performance for copy-move image forgery detection involving similar but genuine objects

Copy-move forgery (CMF) is considered easier to detect than general forgery mechanisms, but detecting it in the presence of multiple similar but genuine scene objects (SGOs) is non-trivial. We study the efficacy of human visual perception for copy-move image forgery detection (CMFD) involving SGOs, and compare the same with machine performance. Via an eye tracking study performed with 16 users where pairs of images (one real and the other tampered) were displayed in either parallel or serial fashion, we make the following observations: (1) Forgery detection is quicker and more accurate when images are spatially aligned and presented serially, so that the tampering is conspicuous. (2) Eye fixations focus on corresponding regions of the real and tampered images, with fewer and more localized fixations noted during serial comparison. (3) A gap is noted between CMFD performance of humans and machines, with each being more sensitive to different tampering factors. Overall, results reveal the need for systematic visual comparisons to distinguish SGOs from forged objects, as well as the promise of a human-machine collaborative framework to this end.
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

Recent approaches in depth-based human activity analysis achieved outstanding performance and proved the effectiveness of 3D representation for classification of action classes. Currently available depth-based and RGB+Dbased action recognition benchmarks have a number of limitations, including the lack of training samples, distinct class labels, camera views and variety of subjects. In this paper we introduce a large-scale dataset for RGB+D human action recognition with more than 56 thousand video samples and 4 million frames, collected from 40 distinct subjects. Our dataset contains 60 different action classes including daily, mutual, and health-related actions. In addition, we propose a new recurrent neural network structure to model the long-term temporal correlation of the features for each body part, and utilize them for better action classification. Experimental results show the advantages of applying deep learning methods over state-of-the-art handcrafted features on the suggested cross-subject and cross-view evaluation criteria for our dataset. The introduction of this large scale dataset will enable the community to apply, develop and adapt various data-hungry learning techniques for the task of depth-based and RGB+D-based human activity analysis.
Epitomic image colorization

Image colorization adds color to grayscale images. It not only increases the visual appeal of grayscale images, but also enriches the information conveyed by scientific images that lack color information. We develop a new image colorization method, epitomic image colorization, which automatically transfers color from the reference color image to the target grayscale image by a robust feature matching scheme using a new feature representation, namely the heterogeneous feature epitome. As a generative model, heterogeneous feature epitome is a condensed representation of image appearance which is employed for measuring the dissimilarity between reference patches and target patches in a way robust to noise in the reference image. We build a Markov Random Field (MRF) model with the learned heterogeneous feature epitome from the reference image, and inference in the MRF model achieves robust feature matching for transferring color. Our method renders better colorization results than the current state-of-the-art automatic colorization methods in our experiments.
PatchMatch-Based Automatic Lattice Detection for Near-Regular Textures

In this work, we investigate the problem of automatically inferring the lattice structure of near-regular textures (NRT) in real-world images. Our technique leverages the PatchMatch algorithm for finding k-nearest-neighbor (kNN) correspondences in an image. We use these kNNs to recover an initial estimate of the 2D wallpaper basis vectors, and seed vertices of the texture lattice. We iteratively expand this lattice by solving an MRF optimization problem. We show that we can discretize the space of good solutions for the MRF using the kNNs, allowing us to efficiently and accurately optimize the MRF energy function using the Particle Belief Propagation algorithm. We demonstrate our technique on a benchmark NRT dataset containing a wide range of images with geometric and photometric variations, and show that our method clearly outperforms the state of the art in terms of both texel detection rate and texel localization score.
Is physics-based liveness detection truly possible with a single image?

Face recognition is an increasingly popular method for user authentication. However, face recognition is susceptible to playback attacks. Therefore, a reliable way to detect malicious attacks is crucial to the robustness of the system. We propose and validate a novel physics-based method to detect images recaptured from printed material using only a single image. Micro-textures present in printed paper manifest themselves in the specular component of the image. Features extracted from this component allows a linear SVM classifier to achieve 2.2% False Acceptance Rate and 13% False Rejection Rate (6.7% Equal Error Rate). We also show that the classifier can be generalizable to contrast enhanced recaptured images and LCD screen recaptured images without re-training, demonstrating the robustness of our
Tracking objects using 3D object proposals

3D object proposals, quickly detected regions in a 3D scene that likely contain an object of interest, are an effective approach to improve the computational efficiency and accuracy of the object detection framework. In this work, we propose a novel online method that uses our previously developed 3D object proposals, in a RGB-D video sequence, to match and track static objects in the scene using shape matching. Our main observation is that depth images provide important information about the geometry of the scene that is often ignored in object matching techniques. Our method takes less than a second in MATLAB on the UW-RGBD scene dataset on a single thread CPU and thus, has potential to be used in low-power chips in Unmanned Aerial Vehicles (UAVs), quadcopters, and drones.
An Automated Estimator of Image Visual Realism Based on Human Cognition

Assessing the visual realism of images is increasingly becoming an essential aspect of fields ranging from computer graphics (CG) rendering to photo manipulation. In this paper we systematically evaluate factors underlying human perception of visual realism and use that information to create an automated assessment of visual realism. We make the following unique contributions. First, we established a benchmark dataset of images with empirically determined visual realism scores. Second, we identified attributes potentially related to image realism, and used correlational techniques to determine that realism was most related to image naturalness, familiarity, aesthetics, and semantics. Third, we created an attributes-motivated, automated computational model that estimated image visual realism quantitatively. Using human assessment as a benchmark, the model was below human performance, but outperformed other state-of-the-art algorithms.