nils-herrmann/clean_scirepeval huggingface dataset:
Real-time facial landmarks tracking using active shape model and LK optical flow

Active Shape Models (ASM) is a generative model widely used to model faces. ASM has been successfully used for face and emotion recognition, and it is one of the state-of-the-art approaches because of its efficiency and representational power. Although widely employed, applying only ASM is not adequate for the practical applications, because positions of the facial landmarks are unstably extracted like jittering movements in the sequential frames, which degrades the performance of the applications. In this paper, we propose a framework for real-time facial landmarks extraction and tracking using ASM and Lucas-Kanade (LK) optical flow which is considered desirable to estimate time-varying geometric parameters in sequential dynamic images of face. In addition, we introduce a straightforward method to avoid failure to extract the facial landmarks by occlusion using the positions of the extracted landmarks by ASM and tracked by LK optical flow. Experimental results validate our approach.
Specular Reflection Separation Using Dark Channel Prior

We present a novel method to separate specular reflection from a single image. Separating an image into diffuse and specular components is an ill-posed problem due to lack of observations. Existing methods rely on a specular-free image to detect and estimate specularity, which however may confuse diffuse pixels with the same hue but a different saturation value as specular pixels. Our method is based on a novel observation that for most natural images the dark channel can provide an approximate specular-free image. We also propose a maximum a posteriori formulation which robustly recovers the specular reflection and chromaticity despite of the hue-saturation ambiguity. We demonstrate the effectiveness of the proposed algorithm on real and synthetic examples. Experimental results show that our method significantly outperforms the state-of-the-art methods in separating specular reflection.
Terrain mapping for a roving planetary explorer

The authors are prototyping a legged vehicle, the Ambler, for an exploratory mission on another planet, conceivably Mars, where it is to traverse uncharted areas and collect material samples. They describe how the rover can construct from range imagery a geometric terrain representation, i.e., elevation map that includes uncertainty, unknown areas, and local features. First, they present an algorithm for constructing an elevation map from a single range image. By virtue of working in spherical-polar space, the algorithm is independent of the desired map resolution and the orientation of the sensor, unlike algorithms that work in Cartesian space. Secondly, the authors present a two-stage matching technique (feature matching followed by iconic matching) that identifies the transformation T corresponding to the vehicle displacement between two viewing positions. Thirdly, to support legged locomotion over rough terrain, they describe methods for evaluating regions of the constructed elevation maps as footholds.<<ETX>>
Change detection using a statistical model of the noise in color images

We present a novel change detection method using a statistical model of the image noise. Most change detection methods are based on gray-level images. However, color images can provide much richer scene information. One major problem to use the color images in change detection is how to combine three components in color space as a detection cue. We use the Euclidean color distance of three channels to measure the difference between two consecutive images. Specifically, we present a new noise model for each color channel. Through this modeling we can estimate the distribution of the Euclidean color distance for unchanged regions. We can find the optimal threshold to detect changes using this estimated distribution. Although we use the optimal threshold, inevitably there may be false classifications. To reject these erroneous cases, we adopt the graph cuts method that efficiently minimizes the global energy, which takes into account the effect of neighboring pixels.
Bayesian filtering for localization using decoupled visual measurements

In this paper, we present a particle-filter-based localization framework with decoupled visual measurements (image features) for process and measurement models. Thus our approach enables using camera-based motion estimation while achieving the independence between the process noise and the measurement noise in the Bayesian filtering framework. In addition, we alternately perform sequential and global localization on the basis of the marginal likelihood in order to avoid severe errors caused by incorrect data association.
Fluttering Pattern Generation Using Modified Legendre Sequence for Coded Exposure Imaging

Finding a good binary sequence is critical in determining the performance of the coded exposure imaging, but previous methods mostly rely on a random search for finding the binary codes, which could easily fail to find good long sequences due to the exponentially growing search space. In this paper, we present a new computationally efficient algorithm for generating the binary sequence, which is especially well suited for longer sequences. We show that the concept of the low autocorrelation binary sequence that has been well exploited in the information theory community can be applied for generating the fluttering patterns of the shutter, propose a new measure of a good binary sequence, and present a new algorithm by modifying the Legendre sequence for the coded exposure imaging. Experiments using both synthetic and real data show that our new algorithm consistently generates better binary sequences for the coded exposure problem, yielding better deblurring and resolution enhancement results compared to the previous methods for generating the binary codes.
Control strategies for a humanoid robot to drive and then egress a utility vehicle for remote approach

This paper proposes strategies for the driving and egress of a vehicle with a humanoid robot. To drive the vehicle, the RANSAC method was used to detect obstacles, and the Wagon model was used to control the steering and velocity of the vehicle with only a limited number of sensors which were installed on the humanoid robot. Additionally, a manual tele-operating method was used with the lane projection technique. For the egress motion, gain override and the Cartesian position/force control technique were used to interact with the vehicle structure. To overcome the disadvantages of a highly geared manipulator, a special technique was used that included modelled friction compensation and a non-complementary switching mode. DRC-HUBO+ used the proposed method to perform a vehicle driving and egress task in the DRC finals 2015.
Multispectral pedestrian detection: Benchmark dataset and baseline

With the increasing interest in pedestrian detection, pedestrian datasets have also been the subject of research in the past decades. However, most existing datasets focus on a color channel, while a thermal channel is helpful for detection even in a dark environment. With this in mind, we propose a multispectral pedestrian dataset which provides well aligned color-thermal image pairs, captured by beam splitter-based special hardware. The color-thermal dataset is as large as previous color-based datasets and provides dense annotations including temporal correspondences. With this dataset, we introduce multispectral ACF, which is an extension of aggregated channel features (ACF) to simultaneously handle color-thermal image pairs. Multi-spectral ACF reduces the average miss rate of ACF by 15%, and achieves another breakthrough in the pedestrian detection task.
Stereo Matching with Color and Monochrome Cameras in Low-Light Conditions

Consumer devices with stereo cameras have become popular because of their low-cost depth sensing capability. However, those systems usually suffer from low imaging quality and inaccurate depth acquisition under low-light conditions. To address the problem, we present a new stereo matching method with a color and monochrome camera pair. We focus on the fundamental trade-off that monochrome cameras have much better light-efficiency than color-filtered cameras. Our key ideas involve compensating for the radiometric difference between two cross-spectral images and taking full advantage of complementary data. Consequently, our method produces both an accurate depth map and high-quality images, which are applicable for various depth-aware image processing. Our method is evaluated using various datasets and the performance of our depth estimation consistently outperforms state-of-the-art methods.
Vision system and depth processing for DRC-HUBO+

This paper presents a vision system and a depth processing algorithm for DRC-HUBO+, the winner of the DRC finals 2015. Our system is designed to reliably capture 3D information of a scene and objects and to be robust to challenging environment conditions. We also propose a depth-map upsampling method that produces an outliers-free depth map by explicitly handling depth outliers. Our system is suitable for robotic applications in which a robot interacts with the real-world, requiring accurate object detection and pose estimation. We evaluate our depth processing algorithm in comparison with state-of-the-art algorithms on several synthetic and real-world datasets.
Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision

Robust Principal Component Analysis (RPCA) via rank minimization is a powerful tool for recovering underlying low-rank structure of clean data corrupted with sparse noise/outliers. In many low-level vision problems, not only it is known that the underlying structure of clean data is low-rank, but the exact rank of clean data is also known. Yet, when applying conventional rank minimization for those problems, the objective function is formulated in a way that does not fully utilize a priori target rank information about the problems. This observation motivates us to investigate whether there is a better alternative solution when using rank minimization. In this paper, instead of minimizing the nuclear norm, we propose to minimize the partial sum of singular values. The proposed objective function implicitly encourages the target rank constraint in rank minimization. Our experimental analyses show that our approach performs better than conventional rank minimization when the number of samples is deficient, while the solutions obtained by the two approaches are almost identical when the number of samples is more than sufficient. We apply our approach to various low-level vision problems, e.g. high dynamic range imaging, photometric stereo and image alignment, and show that our results outperform those obtained by the conventional nuclear norm rank minimization method.
Deep representation of industrial components using simulated images

In this paper, we present a visual learning framework to retrieve a 3D model and estimate its pose from a single image. To increase the quantity and quality of training data, we define our simulation space in the near infrared (NIR) band, and utilize the quasi-Monte Carlo (MC) method for scalable photorealistic rendering of manufactured components. Two types of convolutional neural network (CNN) architectures are trained over these synthetic data and a relatively small amount of real data. The first CNN model seeks the most discriminative information and uses it to classify industrial components with fine-grained shape attributes. Once a 3D model is identified, one of the category-specific CNNs is tested for pose regression in the second phase. The mixed data for learning object categories is useful in domain adaptation and attention mechanism in our system. We validate our data-driven method with 88 component models, and the experimental results are qualitatively demonstrated. Also, the CNNs trained with various conditions of mixed data are quantitatively analyzed.
Hybrid vision-based SLAM coupled with moving object tracking

In this paper we propose a hybrid vision-based SLAM and moving objects tracking (vSLAMMOT) approach. This approach tightly combines two key methods: a superpixel-based segmentation to detect moving objects and a Rao-Blackwellized Particle Filter to estimate a stereo-vision-based SLAM posterior. Most successful methods perform vision-based SLAM (vSLAM) and track moving objects independently. However, we pose both vSLAM and moving object tracking as a single correlated problem to leverage the performance. Our approach estimates the relative camera motion using the previous tracking result, and then detects moving objects from the estimated camera motion recursively. Moving superpixels are detected by a Markov Random Field (MRF) model which uses spatial and temporal information of the moving objects. We demonstrate the performance of the proposed approach for vSLAMMOT using both synthetic and real datasets and compare the performance with other methods.
Automatic closed eye correction

On a large group picture, having all people open their eyes can turn out to be a difficult task for photographers. Therefore, in this paper, we describe an original method to automatically correct closed eyes on everyday pictures. For this aim, we explore the combination possibilities of (1) Active Shape Model (ASM) to detect facial features, such as eyes, nose and head shape, and (2) Poisson editing to clone open eyes seamlessly. To improve the performance of seamless cloning, we suggest a pre-processing method that adjusts skin luminosity between two pictures. A nearest neighbor-based search to find the best suited pair of eyes among a set of donor candidates is also presented. We applied the proposed algorithm on several pictures and obtained very natural results, which demonstrates the validity of our approach.
Behavior-based Intelligent Robot In Dynamic Indoor Environments

We present a navigation system using multiple sensors for unknown and dynamic indoor environments. To achieve the robustness and flexibility of the mobile robot, we propose a new behavior-based architecture with three groups of clustered (reflexive, purposive, and adaptive) agents that realizes both efficiency in attaining the mission of the robot and robustness against the various kinds or failures that may occur in a dynamic environment. Basic behaviors required for navigation, such as, avoiding obstacles, moving towards free space, and following targets, are redundantly developed as agents and combined in the behavior-based system architecture.
2D/3D virtual face modeling

We propose a novel and simple framework that solves two popular problems in digital photography: 2D face synthesis and 3D face modeling. 2D face synthesis aims at creating a new face, usually by mixing two or more portraits. We extend this notion to the combination of human and statue faces. The goal of 3D face modeling is to reconstruct a face in three dimensions from one or several images. These two tasks are often treated as separate problems although they both consider face modeling. In this paper, we propose a unified and general framework for both 2D and 3D cases that runs in a fully automatic manner. Our work also creates stereoscopic views for entertainment 3D display. Experimental results and subjective tests have confirmed the validity of our approach.
Appearance-based gaze estimation using kinect

Human gaze tracking has gathered much attention due to its capability to detect intuitive attention. Appearance-based methods can work with a single camera in ordinary conditions to track human gaze. An effective way to generate eye appearances is proposed using the Kinect. The head pose information is obtained from the Kinect after a series of calibrations. The Eye Appearance features are collected through ASM and KLT feature tracker. With 23 training samples, the error is found to be 1.07°. This paper proposes an efficient scheme for gaze tracking using a single Kinect device.
A visual tracking algorithm by integrating rigid model and snakes

This paper presents a robust vision algorithm for tracking the boundary of an object with an arbitrary shape by using monocular image sequences. This method consists of a curve registration based optimization technique and a deformable contour model ("snakes") for the global and the local motion estimations, respectively. By combining techniques, we overcome, among other problems, inaccurate estimate of motion parameters in the curve registration method (which apparently only occur when a rigid or a flexible object is tracked), and the "local position variation" of the deformable contour model, variations, which are due to noisy images and/or complex backgrounds. The curve registration method uses an iterative algorithm to find the minimum normal distance between two curves, one before motion and the corresponding curve after it. Snakes overcome the limitation of the curve registration method, which suffers from the inaccuracy of motion models. We also propose an internal force, which increases local robustness of the deformable contour to background noise. By using the refined snakes' control points, the global update of the previous curve is performed for the re-location of the registered curve. Additionally, we integrate the geometric invariant value of the boundary contour and the curve registration method to solve the occlusion problem in visual tracking. The proposed method is validated through experiments on real images.
Fused robot pose estimation using embedded and external cameras

Recently, the interest for autonomous driving and unmanned aerial vehicle has drastically increased in the field of robotics. This very clear trend is now expending to everyday life devices, such as lawn mower. In this context we present an automatic lawn mower vision system capable of estimating its absolute pose using both a monocular camera embedded on the robot and an external camera observing the environment where the robot is functioning. This configuration is particularly desirable since the pose estimation of the robot on the lawn is challenging. This can be explained by the fact that the lawn contains only a few feature points which can be tracked reliably. In order to overcome this problem, we propose the use of an external camera to simplify the localization problem using a marker on the robot. Nonetheless, the pose estimation from the external camera alone is not sufficiently accurate for far distances. To improve its precision, we fuse the pose estimation from the surveillance camera with the relative motion of the robot computed from an on-board fisheye camera. We demonstrate the efficiency of the proposed method using real data captured by a wireless synchronized camera network.
Rao-Blackwellized particle filter for Gaussian mixture models and application to visual tracking

One of the most important problems in visual tracking is how to incrementally update the appearance model because the appearance of a target object can be easily changed with time when the target is a deformable object or it is moving under varying illumination conditions. To solve these problems, we present a Rao-Blackwellized particle filter (RBPF)-based object tracking algorithm with the adaptive appearance model represented by a Gaussian mixture model (or a mixture of Gaussians model) because a single Gaussian reveals limitations in modeling the target appearance when observations are corrupted by occlusion or the tracking error. We demonstrate the robustness of the proposed method using well-known databases, such as the CAVIAR and the PETS databases.
Graph-based shape matching for deformable objects

In this paper, we propose a graph-based shape matching method for deformable objects. In our approach, a graph is generated from an over-segmented input image, and the shape matching problem is treated as finding an optimal cycle in the graph. Given a shape template and a graph generated from the input, a product graph is generated to consider every possible correspondence between graph edges and template sub-parts. Because the proposed approach can estimate reasonable correspondences between a target object and a template, it is possible to extract the target object robustly in the presence of shape deformation and background clutter. The experiments on various examples are also presented to verify the performance of proposed method.
Robust feature matching for loop closing and localization

Recently, many vision-based SLAM methods have achieved good results using visual features. However, most algorithms suffer from the accumulated error that inevitably occurs. In this paper, we propose a robust loop detection method by matching image features between the incoming image and key-frame images saved in SLAM. Loop detection is a task of deciding whether a robot has returned to a previously visited area or not. Because a camera is unlikely to have the same pose when a robot revisits the place where it previously encountered, it is crucial to match the features under the different views of the scene. In contrast with view-invariant features, it is hard to match corner points in that situation due to the large variation of neighboring pixels. So we present the robust corner matching method under the view changes. Experimental results demonstrate the capability of the loop closing and mobile robot localization under the different views using the proposed method.
Depth estimation from light field cameras

We present a 3D reconstruction method to estimate depth maps using light-field camera. The estimated depth is widely used for photographic editing. The proposed algorithm, as a feature-based method, provides high quality depth map. First, the distinctive features are extracted from reference image, and the correspondences are tracked. Second, the depth values for each points are calculated by small angle approximated bundle adjustment. Finally, the whole depth map is estimated by simple depth propagation method. To show the effectiveness of our algorithm, we compare our results with the state-of-the-art algorithm.
High resolution terrain map from multiple sensor data

Describes a terrain mapping 3D vision system to build a high resolution terrain map from multiple range images and a digital elevation model (DEM). To build a composite map of the environment from multiple sensor data, the terrain mapping system needs a representation of the terrain that must be appropriate for multiple sensor data. Building a composite terrain map also requires estimating motion between sensor views and merging these views into a composite map. The terrain representation described consists of a grid-based representation, called elevation map. The authors develop the locus method to build elevation maps from range images. The locus method uses a model of the sensor to interpolate at arbitrary resolution without making any assumptions on the terrain shape other than the continuity of the surface. They also present a pixel-based or iconic terrain matching algorithm to estimate the vehicle motion from a sequence of range images. This terrain matching method uses the locus method to solve correspondence and occlusion problems. Comprehensive test results using a long sequence of range images and a DEM for rugged outdoor terrain are given.<<ETX>>
Rich feature hierarchies from omni-directional RGB-DI information for pedestrian detection

In this paper, we propose an omni-directional pedestrian detection method from color, depth, and laser intensity (RGB-DI) information by fusing two different sensors, catadioptric camera and 3D LiDAR scanner. Our method is based on the use of Regions with Convolutional Neural Network (R-CNN) features, which is known as the state-of-the-art object detection method at this moment. The problem of R-CNN is that it takes long computation times over omni-directional searches. By fusing two sensors, we reduced the number of candidate regions and the whole computation time under half, and achieved better performances in the outdoor environment.