nils-herrmann/clean_scirepeval huggingface dataset:
Interactive visualization technique for truthful color reproduction in spatial augmented reality applications

Spatial augmented reality is especially interesting for the design process of a car, because a lot of virtual content and corresponding real objects are used. One important issue in such a process is that the designer can trust the visualized colors on the real object, because design decisions are made on basis of the projection. In this article, we present an interactive visualization technique which is able to exactly compute the RGB values for the projected image, so that the resulting colors on the real object are equally perceived as the real desired colors. Our approach computes the influences of the ambient light, the material, the pose and the color model of the projector to the resulting colors of the projected RGB values by using a physically-based computation. This information allows us to compute the adjustment for the RGB values for varying projector positions at interactive rates. Since the amount of projectable colors does not only depend on the material and the ambient light, but also on the pose of the projector, our method can be used to interactively adjust the range of projectable colors by moving the projector to arbitrary positions around the real object. The proposed method is evaluated in a number of experiments.
Surface segmentation and modeling of 3D polygonal objects from stereoscopic image pairs

An approach to automatically generate 3D polygonal models from stereoscopic image pairs of piecewise planar objects is presented. Dense disparity maps are computed by constrained epipolar block matching. Local surface orientations are computed from the quantized disparity map using a spline approximation under explicit consideration of quantization noise. The local surface orientation is then clustered into regions of similar surface orientation to find the dominant object planes. A photo realistic 3D polygonal model of the object is constructed by fitting planar polygons to the surfaces and by mapping original image texture to the model.
ToF-sensors: New dimensions for realism and interactivity

A growing number of applications depend on accurate and fast 3D scene analysis. Examples are object recognition, collision prevention, 3D modeling, mixed reality, and gesture recognition. The estimation of a range map by image analysis or laser scan techniques is still a time- consuming and expensive part of such systems. A lower-priced, fast and robust alternative for distance measurements are time-of-flight (ToF) cameras. Recently, significant improvements have been made in order to achieve low-cost and compact ToF-devices, that have the potential to revolutionize many fields of research, including computer vision, computer graphics and human computer interaction (HCI). These technologies are starting to have an impact on research and commercial applications. The upcoming generation of ToF sensors, however, will be even more powerful and will have the potential to become "ubiquitous geometry devices" for gaming, web-conferencing, and numerous other applications. This paper will give an account of some recent developments in ToF-technology and will discuss applications of this technology for vision, graphics, and HCI.
Shape adaptation for modelling of 3D objects in natural scenes

Rigid 3D objects were modelled automatically from an image sequence taken by a camera that was rotated around the object. The image sequence was recorded using a calibrated camera which allows one to measure the camera positions and to estimate the true object size. The 3D object shape was obtained in two steps. The object silhouettes were employed to find the enclosing volume of the object. The volume was converted into a flexible surface representation and the 3D shape was refined based on the texture information of the object surface. Texture mapping was applied to generate a highly realistic 3D model of the object.<<ETX>>
Network protocol for interaction and scalable distributed visualization

In the last decade the visualization of virtual environments and interaction within was possible only with specialized hardware. This hardware was very expensive, had a lack of scalability and used specific protocols, buses, networks for communication and specialized graphics hardware for visualization. We present a simple protocol for synchronized distributed scalable visualization and interaction processing in distributed environments. The goal of our protocol is that it can run with standard network protocols like TCP/IP and UDP and standard hardware. It is easy to integrate our protocol into every OpenGL based visualization but the approach is not restricted to OpenGL. These design issues can be used to integrate every standard acceleration for visualization like distributed rendering. We focus on synchronized multiview visualization.
A Mobile Augmented Reality System with Distributed Tracking

For AR applications the 3D position and direction of the users view have to be determined in real time. At the same time, the augmentation, one or more 3D objects, have to be rendered. This article describes a distributed mobile AR-System designed for industrial service applications. By distributing the different tasks involved in AR, the computational load of the portable system can be minimized such that a small and lightweight computer can be used. The portable computer is connected to a backend server via WLAN. Minimisation of the bandwidth usage of the system ensures that 20 or more systems can operate on one standard WLAN access point. The main focus of this article is to minimise the computational load of the mobile system and the network load simultaneously.
Physically-based augmentation of real objects with virtual content under the influence of ambient light

In this article, we introduce a new method for the augmentation of a real object with virtual content under the influence of ambient light. For this purpose, we use a physically-based simulation and spectral data which is recorded separately for each of the components: projector, ambient light and real object. Then the simulation computes the influence of every component to the scene and uses the results to compute an adjusted virtual content, which can then be displayed by a real projector in the given situation. The adjustment of the projection can therefore be computed by considering the full dynamic range of the scene and that of the virtual content. Additionally, the virtual content is visualized with informations from the environment and therefore hidden objects or reflections can be integrated into the projection. The proposed methods and the physically-based simulation are validated and visualized through different experiments.
3D reconstruction based on underwater video from ROV Kiel 6000 considering underwater imaging conditions

This work presents a system for 3D reconstruction from underwater images or video. Aside from a camera in an underwater housing, no special equipment is required. However, if navigational data is available, it is utilized in the algorithm. The algorithm is in essence a classical structure from motion approach, which is adapted to account for the special imaging conditions. Hence, there is no need for the camera to follow a specialized trajectory. Adaptions to the underwater imaging environment include a special filtering of background and floating particles, which allows a robust estimation of the camera poses and a sparse set of 3D points. Based on the estimated camera track, dense image correspondence computation enables building a detailed 3D surface model. Once the 3D surface model is completed, the colors of the texture are corrected by a physical model for underwater light propagation, allowing to view the model without the effects of scattering and attenuation or to simulate the effects of water on light in a 3D viewer.
Distributed interaction processing and visualization of 3D scenes in realtime

A distributed realtime system for immersive visualization is presented which uses distributed interaction for control. We have focused on a network architecture for distributed interaction processing for multiple devices and distributed visualization. Furthermore we also discussed in detail the user tracking with fixed and pan-tilt-zoom cameras. One of our goals is the use of standard hardware and standard network protocols for the system. For the distributed realtime visualization we use consumer graphics hardware only.
Camera calibration with known rotation

We address the problem of using external rotation information with uncalibrated video sequences. The main problem addressed is, what is the benefit of the orientation information for camera calibration? It is shown that in case of a rotating camera the camera calibration problem is linear even in the case that all intrinsic parameters vary. For arbitrarily moving cameras the calibration problem is also linear but underdetermined for the general case of varying all intrinsic parameters. However, if certain constraints are applied to the intrinsic parameters the camera calibration can be computed linearly. It is analyzed which constraints are needed for camera calibration of freely moving cameras. Furthermore we address the problem of aligning the camera data with the rotation sensor data in time. We give an approach to align these data in case of a rotating camera.
Conjugate rotation: Parameterization and estimation from an affine feature correspondence

When rotating a pinhole camera, images are related by the infinite homography KRK-1, which is algebraically a conjugate rotation. Although being a very common image transformation, e.g. important for self-calibration or panoramic image mosaicing, it is not completely understood yet. We show that a conjugate rotation has 7 degrees of freedom (as opposed to 8 for a general homography) and give a minimal parameterization. To estimate the conjugate rotation, authors traditionally made use of point correspondences, which can be seen as local zero order Taylor approximations to the image transformation. Recently however, affine feature correspondences have become increasingly popular. We observe that each such affine correspondence now provides a local first order Taylor approximation, which has not been exploited in the context of geometry estimation before. Using those two novel concepts above, we finally show that it is possible to estimate a conjugate rotation from a single affine feature correspondence under the assumption of square pixels and zero skew. As a byproduct, the proposed algorithm directly yields rotation, focal length and principal point.
A Comparison of PMD-Cameras and Stereo-Vision for the Task of Surface Reconstruction using Patchlets

Recently real-time active 3D range cameras based on time-of-flight technology (PMD) have become available. Those cameras can be considered as a competing technique for stereo-vision based surface reconstruction. Since those systems directly yield accurate 3d measurements, they can be used for benchmarking vision based approaches, especially in highly dynamic environments. Therefore, a comparative study of the two approaches is relevant. In this work the achievable accuracy of the two techniques, PMD and stereo, is compared on the basis of patch-let estimation. As patchlet we define an oriented small planar 3d patch with associated surface normal. Least-squares estimation schemes for estimating patchlets from PMD range images as well as from a pair of stereo images are derived. It is shown, how the achivable accuracy can be estimated for both systems. Experiments under optimal conditions for both systems are performed and the achievable accuracies are compared. It has been found that the PMD system outperformed the stereo system in terms of achievable accuracy for distance measurements, while the estimation of normal direction is comparable for both systems.
A linear method for recovering the depth of Ultra HD cameras using a kinect V2 sensor

Depth-Image-Based Rendering (DIBR) is a mature and important method for making free-viewpoint videos. As for the study of the DIBR approach, on the one hand, most of current research focuses on how to use it in systems with low resolution cameras, while a lot of Ultra HD rendering devices have been launched into markets. On the other hand, the quality and accuracy of the depth image directly affects the final rendering result. Therefore, in this paper we try to make some improvements on solving the problem of recovering the depth information for Ultra HD cameras with the help of a Kinect V2 sensor. To this end, a linear least squares method is proposed, which recovers the rigid transformation between a Kinect V2 and an Ultra HD camera, using the depth information from the Kinect V2 sensor. In addition, a non-linear coarse-to-fine method, which is based on Sparse Bundle Adjustment (SBA), is compared with this linear method. Experiments show that our proposed method performs better than the non-linear method for the Ultra HD depth image recovery both in computing time and precision.
Refractive Structure-from-Motion on Underwater Images

In underwater environments, cameras need to be confined in an underwater housing, viewing the scene through a piece of glass. In case of flat port underwater housings, light rays entering the camera housing are refracted twice, due to different medium densities of water, glass, and air. This causes the usually linear rays of light to bend and the commonly used pinhole camera model to be invalid. When using the pinhole camera model without explicitly modeling refraction in Structure-from-Motion (SfM) methods, a systematic model error occurs. Therefore, in this paper, we propose a system for computing camera path and 3D points with explicit incorporation of refraction using new methods for pose estimation. Additionally, a new error function is introduced for non-linear optimization, especially bundle adjustment. The proposed method allows to increase reconstruction accuracy and is evaluated in a set of experiments, where the proposed method's performance is compared to SfM with the perspective camera model.
Multi-camera real-time depth estimation with discontinuity handling on PC graphics hardware

This paper describes a system for dense depth estimation from multiple color images in real-time. Our algorithm runs almost entirely on standard graphics hardware, leaving the main CPU free for other tasks such as image capture and higher level recognition. We follow a plane-sweep approach extended by truncated SSD scores, spatially shiftable windows and best camera selection to handle discontinuities. We do not need specialized hardware and exploit the computational power of freely programmable PC GPU hardware. Dense depth maps are computed with up to 20 fps.
Improving depth discontinuities for depth-based 3DTV production

We introduce a scheme for the production of LDV (Layered Depth Video) content, based on capturing from a hybrid camera system, consisting of two time of flight and five CCD cameras and introduce a post production step, based on Grab - Cut segmentation, which significantly improves the quality of the end result. The proposed improvement step can be done fully automatically, based on initialization through estimated depth images.
A Novel Self-Calibration Method for a Stereo-ToF System Using a Kinect V2 and Two 4K GoPro Cameras

A new light-field movie capture device using a Kinect V2 sensor and two 4K GoPro cameras is presented in this paper. Due to the uncontrollable tilt of the Kinect V2 camera, it is hard to obtain a constant rigid transformation between the stereo- and ToF-camera systems. To this end, a novel self-calibration method is proposed, which takes advantage of the geometric constraints from the scene and the cameras. Specifically, a camera orientation approximation approach is utilized to estimate the rigid transformation of the stereo-ToF system based on reliable point pairs filtered by the geometric constraints. Besides, a depth correction step is exploited to improve the depth accuracy of the Kinect V2 sensor. Moreover, a depth fusion strategy for the stereo- and ToF-depth data is proposed to provide more accurate depth images in 4K resolution. Experimental results demonstrate the effectiveness of the proposed depth correction step, stereo-ToF calibration method and depth fusion strategy.
A novel kinect V2 registration method for large-displacement environments using camera and scene constraints

In a lot of multi-Kinect V2-based systems, the registration of these Kinect V2 sensors is an important step which directly affects the system precision. The coarse-to-fine method using calibration objects is an effective way to solve the Kinect V2 registration problem. However, for the registration of Kinect V2 cameras with large displacements, this kind of method may fail. To this end, a novel Kinect V2 registration method, which is also based on the coarse-to-fine framework, is proposed by using camera and scene constraints. Specifically, in the coarse estimation stage, scene constraints are explored using off-the-shelf feature point detectors and camera constraints are explored using homography and fundamental matrices. In the estimation refinement stage, an Iterative Closest Point (ICP)-based point cloud registration method is utilized. Experimental results show that the proposed Kinect V2 registration method using camera and scene constraints performs much better in precision than using calibration objects in the large-displacement environment.
Image-based rendering of complex scenes from a multi-camera rig

Image-based rendering is a method to synthesise novel views from a set of given real images. Two methods to extrapolate novel views of complex scenes with occlusions and large depth discontinuities from images of a moving uncalibrated multi-camera rig are discribed. The real camera viewpoints are calibrated from the image data and dense depth maps are estimated for each real view. Novel views are synthesised from this representation with view-dependent image-based rendering techniques at interactive rates. Since the 3-D scene geometry is available in this approach, it is well suited for mixed reality applications where synthetic 3-D objects are seamlessly embedded in the novel view.
Lens model selection for a markerless AR tracking system

This paper describes a visual markerless real-time tracking system for augmented reality applications. The system uses a firewire camera with a fisheye lens mounted at 10 fps. Visual tracking of 3D scene points is performed simultaneously with 3D camera pose estimation without any prior scene knowledge. All visual-geometric data is acquired using a structure-from-motion approach. The lens selection was driven by research results that show the superiority of a fisheye lens to a standard perspective lens for this approach. 2D features in the hemispherical image are tracked using a 2D point tracker. Based on the feature tracks, 3D camera ego-motion and 3D features are estimated.
Finite-Impulse-Response Digital Differentiators for Midband Frequencies Based on Maximal Linearity Constraints

Maximally linear digital differentiators (DDs) are known for high accuracy in narrow frequency bands centered at the frequency for which they are designed. In this paper, designs of DDs of odd and even lengths having maximal linearity at the middle of the frequency band are presented. Applying the maximal linearity constraints to the magnitude response of a differentiator gives a system of linear equations, which can be solved for the impulse response coefficients of the differentiator. It is observed that the coefficient matrices of these equations are Vandermonde matrices, and this helps in finding the solution of the equations in closed form. Design examples are presented to show the accuracy of the presented designs, and it is observed that even-length designs are more accurate in a significantly wider frequency band as compared with odd-length designs
Multiple Exposure Fusion for High Dynamic Range Image Acquisition

A multiple exposure fusion to enhance the dynamic range of an image is proposed. The construction of high dynamic range images (HDRIs) is performed by combining multiple images taken with different exposures and estimating the irradiance value for each pixel. This is a common process for HDRI acquisition. During this process, displacements of the images caused by object movements often yield motion blur and ghosting artifacts. To address the problem, this paper presents an efficient and accurate multiple exposure fusion technique for the HDRI acquisition. Our method simultaneously estimates displacements and occlusion and saturation regions by using maximum a posteriori estimation and constructs motion-blur-free HDRIs. We also propose a new weighting scheme for the multiple image fusion. We demonstrate that our HDRI acquisition algorithm is accurate, even for images with large motion.
Fast and stable least-squares approach for the design of linear phase FIR filters

We present a new numerical algorithm for solving the normal equations associated with the least-squares design of linear phase FIR filters. The usual solution methods have a computational complexity of O(N/sup 3/). Moreover, solving the normal equations with Gaussian elimination commonly yields numerical errors, especially when the filter is long. Here, we convert a least-squares method into the problem of constructing a system of orthonormal functions. The proposed design algorithm needs only O(N/sup 2/) computations, and numerical errors can be reduced. Some examples are given to show the performance of the algorithm.
Multiresolution Modeling of 3D Maps

Three-dimensional (3D) urban models, come with huge data size and their simplification is often needed for efficient streaming, rendering and visualization. In this paper, we present a strategy for organization of this data such that the models of reduced resolution can be readily created. Simplification techniques for different types of objects in the urban models are presented and are used to represent them at different levels of detail. Objects are classified based on their importance relative to other objects in the model, and based on this classification, a suitable level of details of each object is determined for inclusion in the model of reduced resolution
High Dynamic Range image tone mapping based on local Histogram Equalization

High Dynamic Range (HDR) images can represent the acquired scene with a greater dynamic range of luminance than classical Low Dynamic Range (LDR) ones. Despite the recent diffusion of some HDR camera models, HDR displays are not yet in the market. For this reason HDR images need to be adapted in order to be properly rendered through conventional devices. This operation mainly consists in a dynamic range compression realized by applying a Tone Mapping Operator (TMO). In this work, a new tone map algorithm, derived from the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique, is presented. With respect to the original CLAHE, in the proposed implementation an adaptive contrast limit and a new strategy for the determination of local tone mapping functions have been introduced. The comparison between the obtained LDR images, and those produced by applying State of the Art TMOs, evidences how the main characteristic of the proposed algorithm is the ability to equally enhance visibility in both dark and bright areas. This could be, for example, a key feature in video surveillance applications and automotive safety camera systems.