nils-herrmann/clean_scirepeval huggingface dataset:
Human body part classification from optical flow

Body part (BP) classification is important in human image analysis. In this paper, we propose an optical flow based pixel-wise BP classifier using random forest (RF) in a monocular video. Running the BP classifier on each optical frame generates noisy mis classifications. We integrate the classifier with a human detector and temporal voting to reduce most of misclassified pixels. The proposed method is evaluated quantitatively on a real world dataset and showed better performance.
Multi-View Object Extraction With Fractional Boundaries

This paper presents an automatic method to extract a multi-view object in a natural environment. We assume that the target object is bounded by the convex volume of interest defined by the overlapping space of camera viewing frustums. There are two key contributions of our approach. First, we present an automatic method to identify a target object across different images for multi-view binary co-segmentation. The extracted target object shares the same geometric representation in space with a distinctive color and texture model from the background. Second, we present an algorithm to detect color ambiguous regions along the object boundary for matting refinement. Our matting region detection algorithm is based on the information theory, which measures the Kullback-Leibler divergence of local color distribution of different pixel bands. The local pixel band with the largest entropy is selected for matte refinement, subject to the multi-view consistent constraint. Our results are high-quality alpha mattes consistent across all different viewpoints. We demonstrate the effectiveness of the proposed method using various examples.
VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition

In this paper, we propose a unified end-to-end trainable multi-task network that jointly handles lane and road marking detection and recognition that is guided by a vanishing point under adverse weather conditions. We tackle rainy and low illumination conditions, which have not been extensively studied until now due to clear challenges. For example, images taken under rainy days are subject to low illumination, while wet roads cause light reflection and distort the appearance of lane and road markings. At night, color distortion occurs under limited illumination. As a result, no benchmark dataset exists and only a few developed algorithms work under poor weather conditions. To address this shortcoming, we build up a lane and road marking benchmark which consists of about 20,000 images with 17 lane and road marking classes under four different scenarios: no rain, rain, heavy rain, and night. We train and evaluate several versions of the proposed multi-task network and validate the importance of each task. The resulting approach, VPGNet, can detect and classify lanes and road markings, and predict a vanishing point with a single forward pass. Experimental results show that our approach achieves high accuracy and robustness under various conditions in realtime (20 fps). The benchmark and the VPGNet model will be publicly available
Noise Robust Depth from Focus Using a Ring Difference Filter

Depth from focus (DfF) is a method of estimating depth of a scene by using the information acquired through the change of the focus of a camera. Within the framework of DfF, the focus measure (FM) forms the foundation on which the accuracy of the output is determined. With the result from the FM, the role of a DfF pipeline is to determine and recalculate unreliable measurements while enhancing those that are reliable. In this paper, we propose a new FM that more accurately and robustly measures focus, which we call the ring difference filter (RDF). FMs can usually be categorized as confident local methods or noise robust non-local methods. RDFs unique ring-and-disk structure allows it to have the advantageous sides of both local and non-local FMs. We then describe an efficient pipeline that utilizes the properties that the RDF brings. Our method is able to reproduce results that are on par with or even better than those of the state-of-the-art, while spending less time in computation.
3-D object recognition using projective invariant relationship by single-view

We propose a new method for recognizing three-dimensional objects using a three-dimensional invariant relationship for a special structure and geometric hashing by single-view. We use a special structure consisting of four co-planar points and any two non-coplanar points with respect to the plane. We derive an invariant relationship for the structure, which is represented by a plane equation. For recognition of 3-D objects using geometric hashing, a set of points on the plane is mapped into a set of points intersecting the plane and the unit sphere, thereby satisfying the invariant relationship. Experiments using 3-D polyhedral objects are carried out to demonstrate the feasibility of our method for 3-D object recognition.
AttentionNet: Aggregating Weak Directions for Accurate Object Detection

We present a novel detection method using a deep convolutional neural network (CNN), named AttentionNet. We cast an object detection problem as an iterative classification problem, which is the most suitable form of a CNN. AttentionNet provides quantized weak directions pointing a target object and the ensemble of iterative predictions from AttentionNet converges to an accurate object boundary box. Since AttentionNet is a unified network for object detection, it detects objects without any separated models from the object proposal to the post bounding-box regression. We evaluate AttentionNet by a human detection task and achieve the state-of-the-art performance of 65% (AP) on PASCAL VOC 2007/2012 with an 8-layered architecture only.
Evaluation of vocabulary trees for localization in robot applications

Vocabulary tree based place recognition is widely used in topological localization and its various applications have been proposed during the past decade. However, the bag-of-words representations from the vocabulary tree, which is trained with fixed training data, are difficult to be optimized to dynamic environments. To solve this problem, an adaptive vocabulary tree has been proposed, but there has been no comparison considering the adaptive properties of the conventional vocabulary tree. This paper provides a performance evaluation of the vocabulary tree and the adaptive vocabulary tree in dynamic scenes. This analysis provides guidance for choosing appropriate vocabulary in robot applications.
Moving object detection and tracking from moving camera

In this paper, we deal multi moving object detection and tracking under moving camera. Moving objects are detected by homography-based motion detection. After moving objects are detected, we apply online-boosting trackers to track moving objects. Because each tracker and detector is measured independently, we integrate two systems into one system. Hence, our algorithm detects and tracks multi moving objects without background modeling. We show experiment results from sequences which are obtained from natural outdoor scene.
Generalized laser three-point algorithm for motion estimation of camera-laser fusion system

This paper presents a new structure-from-motion (SFM) technique called `generalized laser three-point' algorithm. It is designed to estimate the motion of the camera-laser fusion system which consists of a 2D laser sensor and multiple cameras. The laser points are projected onto the images and tracked to other frames to be used as 3D-2D correspondences. However, the typical three-point algorithms cannot estimate the motion of the system if three points are collinear. Using the laser points as 3D points, this case happens frequently if the laser sensor scans a large plane (e.g. open ground). Even in that case, two frames of the laser data are not collinear if the system is moved while it captures the frames. Among three point correspondences required to estimate the motion, we select two points and the other point from different frames. We estimate the relative pose between the frames by solving an 8-degree polynomial equation. The experimental results show that the proposed algorithm is more appropriate for our fusion system than the previous algorithms.
Stereo Matching with the Distinctive Similarity Measure

The point ambiguity owing to the ambiguous local appearances of image points is the one of the main causes making the stereo problem difficult. Under the point ambiguity, local similarity measures are easy to be ambiguous and this results in false matches in ambiguous regions. In this paper, we present the new similarity measure to resolve the point ambiguity problem based on the idea that the distinctiveness, not the interest, is the appropriate criterion for the feature selection under the point ambiguity. The proposed similarity measure named the Distinctive Similarity Measure (DSM) is essentially based on the distinctiveness of image points and the dissimilarity between them, which are both closely related to the local appearances of image points; the distinctiveness of an image point is related to the probability of a mismatch while the dissimilarity is related to the probability of a good match. We verify the efficiency of the proposed DSM by using testbed image sets. Experimental results show that the proposed DSM is very effective and can be easily used for improving the performance of existing stereo methods under the point ambiguity.
Robust Road Marking Detection and Recognition Using Density-Based Grouping and Machine Learning Techniques

This paper presents a robust approach for road marking detection and recognition from images captured by an embedded camera mounted on a car. Our method is designed to cope with illumination changes, shadows, and harsh meteorological conditions. Furthermore, the algorithm can effectively group complex multi-symbol shapes into an individual road marking. For this purpose, the proposed technique relies on MSER features to obtain candidate regions which are further merged using density-based clustering. Finally, these regions of interest are recognized using machine learning approaches. Worth noting, the algorithm is versatile since it does not utilize any prior information about lane position or road space. The proposed method compares favorably to other existing works through a large number of experiments on an extensive road marking dataset.
Robust vision-based autonomous navigation against environment changes

Recently, many vision-based navigation methods have been introduced as an intelligent robot application. However, many of these methods mainly focus on finding an image in the database corresponding to a query image. Thus, if the environment changes, for example, objects moving in the environment, a robot is unlikely to find consistent corresponding points with one of the database images. To handle these problems, we propose a novel motion-based navigation method in contrast with appearance-based approaches. This algorithm is based on motion estimation by a camera to plan the next movement of a robot and robust feature matching to recognize home and destination locations. Experimental results demonstrate the capability of the vision-based autonomous navigation against environment changes.
A stratified approach to metric self-calibration

Camera calibration is essential to many computer vision applications. In practice this often requires cumbersome calibration procedures to be carried out regularly. In the last few years a lot of work has been done on self-calibration of cameras, ranging from weak calibration to metric calibration. It has been shown that a metric calibration of the camera setup (up to scale) was possible based on the rigidity of the scene only. In this paper a stratified approach is proposed which gradually retrieves the metric calibration of the camera setup. Starting from an uncalibrated image sequence the projective calibration is retrieved first. In projective space the plane at infinity is then identified yielding the affine calibration. This is achieved using a constraint which can be formulated between any two arbitrary images of the sequence. Once the affine calibration is known the upgrade to metric is easily obtained through linear equations.
The modulus constraint: a new constraint self-calibration

To obtain a Euclidean reconstruction from images the cameras have to be calibrated. In recent years different approaches have been proposed to avoid explicit calibration. The problem with these methods is that several parameters have to be retrieved at once. Because of the non-linearity of the equations this is not an easy task and the methods often fail to converge. In the's paper a stratified approach is proposed which allows to first retrieve the affine calibration of the camera using the modulus constraint. Having the affine calibration it is easy to upgrade to Euclidean. The important advantage of this method is that only three parameters have to be evaluated at first. From a practical point of view, the major gain is that an affine reconstruction is obtained from arbitrary sequences of views, whereas so far affine reconstruction has been based on pairs of views with a pure translation in between. A short illustration of another application is also given. Once the affine calibration is known, the constraint can be used to retrieve the Euclidean calibration in the presence of a variable focal length.
Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge Project

Future requirements for drastic reduction of CO2 production and energy consumption will lead to significant changes in the way we see mobility in the years to come. However, the automotive industry has identified significant barriers to the adoption of electric vehicles, including reduced driving range and greatly increased refueling times. Automated cars have the potential to reduce the environmental impact of driving, and increase the safety of motor vehicle travel. The current state-of-the-art in vehicle automation requires a suite of expensive sensors. While the cost of these sensors is decreasing, integrating them into electric cars will increase the price and represent another barrier to adoption. The V-Charge Project, funded by the European Commission, seeks to address these problems simultaneously by developing an electric automated car, outfitted with close-to-market sensors, which is able to automate valet parking and recharging for integration into a future transportation system. The final goal is the demonstration of a fully operational system including automated navigation and parking. This paper presents an overview of the V-Charge system, from the platform setup to the mapping, perception, and planning sub-systems.
Advanced three-dimensional television system technologies

We describe the goals of the ATTEST project, which started in March 2002 as part of the Information Society Technologies (IST) programme, sponsored by the European Commission. In the 2-year project, several industrial and academic partners cooperate towards a flexible, 2D-compatible and commercially feasible 3D-TV system-for broadcast environments. An entire 3D-video chain will be developed. We discuss the goals for content creation, coding, transmission, display and the central role that human 3D perception research will play in optimizing the entire chain. The goals include the development of a new 3D camera, algorithms to convert existing 2D-video material into 3D, a 2D-compatible coding and transmission scheme for 3D video using MPEG-2/4/7, and two new autostereoscopic displays. With the combination of industrial and academic partners and the technological progress obtained from earlier 3D projects, we expect to achieve the ATTEST goal of developing the first commercially feasible European 3D-TV broadcast system.
Differential Camera Tracking through Linearizing the Local Appearance Manifold

The appearance of a scene is a function of the scene contents, the lighting, and the camera pose. A set of n-pixel images of a non-degenerate scene captured from different perspectives lie on a 6D nonlinear manifold in Rn. In general, this nonlinear manifold is complicated and numerous samples are required to learn it globally. In this paper, we present a novel method and some preliminary results for incrementally tracking camera motion through sampling and linearizing the local appearance manifold. At each frame time, we use a cluster of calibrated and synchronized small baseline cameras to capture scene appearance samples at different camera poses. We compute a first-order approximation of the appearance manifold around the current camera pose. Then, as new cluster samples are captured at the next frame time, we estimate the incremental camera motion using a linear solver. By using intensity measurements and directly sampling the appearance manifold, our method avoids the commonly-used feature extraction and matching processes, and does not require 3D correspondences across frames. Thus it can be used for scenes with complicated surface materials, geometries, and view-dependent appearance properties, situations where many other camera tracking methods would fail.
Probabilistic 3D occupancy flow with latent silhouette cues

In this paper we investigate shape and motion retrieval in the context of multi-camera systems. We propose a new low-level analysis based on latent silhouette cues, particularly suited for low-texture and outdoor datasets. Our analysis does not rely on explicit surface representations, instead using an EM framework to simultaneously update a set of volumetric voxel occupancy probabilities and retrieve a best estimate of the dense 3D motion field from the last consecutively observed multi-view frame set. As the framework uses only latent, probabilistic silhouette information, the method yields a promising 3D scene analysis method robust to many sources of noise and arbitrary scene objects. It can be used as input for higher level shape modeling and structural inference tasks. We validate the approach and demonstrate its practical use for shape and motion analysis experimentally.
Radiometric alignment of image sequences

Color values in an image are related to image irradiance by a nonlinear function called radiometric response function. Since this function depends on the aperture and the shutter speed, image intensity of a same object may vary during the acquisition of an image sequence due to auto exposure feature of the camera. While this is desirable to make optimal use of the limited dynamic range of most cameras, this causes problems for a number of applications in computer vision. In this paper we propose a method for estimating the radiometric response function and apply it to radiometrically align images so that the color values are consistent for all images of a sequence. Our approach computes the response function, exposure and white balance changes between images (up to some ambiguity) for a moving camera without any prior knowledge about exposures. We show the performance of our algorithm by estimating the response function from synthetic images and also from real world data, using it to radiometrically align the images.
A hierarchical stereo algorithm using dynamic programming

In this paper, a new hierarchical stereo algorithm is presented. The algorithm matches individual pixels in corresponding scanlines by minimizing a cost function. Several cost functions are compared. The algorithm achieves a tremendous gain in speed and memory requirements by implementing it hierarchically. The images are down sampled an optimal number of times and the disparity map of a lower level is used as 'offset' disparity map at a higher level. An important contribution consists of the complexity analysis of the algorithm. It is shown that this complexity is independent of the disparity range. This result is also used to determine the-optimal number of down sample levels. This speed gain results in the ability to use more complex (compute intensive) cost functions that deliver high quality disparity maps. Another advantage of this algorithm is that cost functions can be chosen independent of the optimisation algorithm. Finally, the algorithm was carefully implemented so that a minimal amount of memory is used. It has proven its efficiency on large images with a high disparity range as well as its quality. Examples are given in this paper.
Classification and Pose Estimation of Vehicles in Videos by 3D Modeling within Discrete-Continuous Optimization

This paper presents a framework for classification and pose estimation of vehicles in videos by assuming their given 3D models. We rank possible poses and types for each frame and exploit temporal coherence between consecutive frames for refinement. As a novelty, first, we cast the estimation of a vehicle's pose and type as a solution of a continuous optimization problem over space and time. Due to a non-convexity of this problem, good initial starting points are important. We propose to obtain them by a discrete temporal optimization reaching a global optimum on a ranked discrete set of possible types and poses. Second, to guarantee effectiveness of the proposed discrete-continuous optimization, we present a novel way to efficiently reduce the search space of potential 3D model types and poses for each frame for the discrete optimizer. It avoids common expensive evaluation of all possible discretized hypotheses. The key idea towards efficiency lies in a novel combination of detecting the vehicle, rendering the 3D models, matching projected edges to input images, and using a tree structured Markov Random Field to get fast and globally optimal inference and to force the vehicle follow a feasible motion model in the initial phase. Quantitative and qualitative experiments on a variety of videos with vast variation of vehicle types show superior results to state-of-the-art methods.
Omnidirectional visual obstacle detection using embedded FPGA

For autonomous navigation of Micro Aerial Vehicles (MAVs) in cluttered environments, it is essential to detect potential obstacles not only in the direction of flight but in their entire local environment. While there exist systems that do vision based obstacle detection, most of them are limited to a single perception direction. Extending these systems to a multi-directional sensing approach would exhaust the payload limit in terms of weight and computational power. We present a novel light-weight sensor setup comprising of four stereo heads and an inertial measurement unit (IMU) to perform FPGA-based dense reconstruction for obstacle detection in all directions. As the data-rate scales up with the number of cameras we use an FPGA to perform streaming based tasks in real-time and show a light-weight polar-coordinate map to allow a companion computer to fully process the data of all the cameras and perform obstacle detection in real-time. The system is able to process up to 80 frames per second (fps) freely distributed on the four stereo heads while maintaining a low power budget. The perception system including FPGA, image sensors and stereo mounts is 235 g in weight.