nils-herrmann/clean_scirepeval huggingface dataset:
Model-based analysis of multi-video data

We describe a method to analyze multiple synchronized video streams by making use of a parameterized geometry model of the recorded object. By formulating the task of fitting the model to the images in terms of optimizing silhouette match, we are able to capture the time-varying 3D pose of the object automatically and robustly. To evaluate the energy functional, we exploit the fast image synthesis capabilities of a conventional PC graphics card. The use of an a-priori object model enables us to enforce kinematic constraints as well as temporal coherence, and we obtain a high-quality surface description as output. Suitably modified, the presented technique is also applicable to medical and other image analysis tasks if a parameterized, generic geometry model of the object of interest is available.
Seeing People in Different Light-Joint Shape, Motion, and Reflectance Capture

By means of passive optical motion capture, real people can be authentically animated and photo-realistically textured. To import real-world characters into virtual environments, however, surface reflectance properties must also be known. We describe a video-based modeling approach that captures human shape and motion as well as reflectance characteristics from a handful of synchronized video recordings. The presented method is able to recover spatially varying surface reflectance properties of clothes from multiview video footage. The resulting model description enables us to realistically reproduce the appearance of animated virtual actors under different lighting conditions, as well as to interchange surface attributes among different people, e.g., for virtual dressing. Our contribution can be used to create 3D renditions of real-world people under arbitrary novel lighting conditions on standard graphics hardware.
Shape: A 3D Modeling Tool for Astrophysics

We present a flexible interactive 3D morpho-kinematical modeling application for astrophysics. Compared to other systems, our application reduces the restrictions on the physical assumptions, data type, and amount that is required for a reconstruction of an object's morphology. It is one of the first publicly available tools to apply interactive graphics to astrophysical modeling. The tool allows astrophysicists to provide a priori knowledge about the object by interactively defining 3D structural elements. By direct comparison of model prediction with observational data, model parameters can then be automatically optimized to fit the observation. The tool has already been successfully used in a number of astrophysical research projects.
Geometry-based automatic object localization and 3-D pose detection

Given the image of a real-world scene and a polygonal 3D model of a depicted object, its apparent size, image coordinates, and 3D orientation are autonomously detected. Based on matching silhouette outline to edges in the image, an extensive search in parameter space converges to the best-matching set of parameter values. Apparent object size may a-priori be unknown, and no initial search parameter values need to be provided. Due to its high degree of parallelism, the algorithm is well suited for implementation on graphics hardware to achieve fast object recognition and 3D pose estimation.
A ghosting artifact detector for interpolated image quality assessment

We present a no-reference image quality metric for image interpolation. The approach is capable of detecting blurry regions as well as ghosting artifacts, e.g., in image based rendering scenarios. Based on the assumption that ghosting artifacts can be detected locally, perceived visual quality can be predicted from the amount of regions that are affected by ghosting. Because the approach does not require any reference image, it is very suitable, e.g., for assessing quality of image-based rendering techniques in general settings.
The capturing of turbulent gas flows using multiple Kinects

We introduce the Kinect as a tool for capturing gas flows around occluders using objects of different aerodynamic properties. Previous approaches have been invasive or require elaborate setups including large printed sheets of complex noise patterns and neat lighting. Our method is easier to set up while still producing good results. We show that three Kinects are sufficient to qualitatively reconstruct nonstationary time varying gas flows in the presence of occluders.
Video-Driven Animation of Human Body Scans

We present a versatile, fast and simple framework to generate animations of scanned human characters from input multiview video sequences. Our method is purely mesh-based and requires only a minimum of manual interaction. The proposed algorithm implicitly generates realistic body deformations and can easily transfer motions between human subjects of completely different shape and proportions. We feature a working prototype system that demonstrates that our method can generate convincing lifelike character animations from marker-less optical motion capture data.
Real-time microfacet billboarding for free-viewpoint video rendering

We present a hardware-accelerated method for video-based rendering relying on an approximate model of scene geometry. Our goal is to render high-quality views of the scene from arbitrary viewpoints in real-time, using as input synchronized video-streams from only a small number of calibrated cameras distributed around the scene. Despite the fact that only a very coarse geometry reconstruction is possible, our rendering approach based on textured billboards preserves the details present in the source images. By exploiting multi-texturing and blending facilities of modern graphics cards, we achieve real-time frame-rates on current off-the-shelf hardware. One of the possible applications of our algorithm is to use it in an inexpensive system to display 3D-videos, which the user can watch from an interactively chosen viewpoint, or ultimately even live 3D-television.
A graphics hardware implementation of the generalized Hough transform for fast object recognition, scale, and 3D pose detection

The generalized Hough transform constitutes a wellknown approach to object recognition and pose detection. To attain reliable detection results, however, a very large number of candidate object poses and scale factors need to be considered. We employ an inexpensive, consumer-market graphics-card as the "poor man's" parallel processing system. We describe the implementation of a fast and enhanced version of the generalized Hough transform on graphics hardware. Thanks to the high bandwidth of on-board texture memory, a single pose can be evaluated in less than 3 ms, independent of the number of edge pixels in the image. From known object geometry, our hardware-accelerated generalized Hough transform algorithm is capable of detecting an object's 3D pose, scale, and position in the image within less than one minute. A good pose estimation is even delivered in less than 10 seconds.
Global Depth from Epipolar Volumes--A General Framework for Reconstructing Non-Lambertian Surfaces

Using epipolar image analysis in the context of the correspondence finding problem in depth reconstruction has several advantages. One is the elegant incorporation of prior knowledge about the scene or the surface reflection properties into the reconstruction process. The proposed framework in conjunction with graph cut optimization is able to reconstruct also highly specular surfaces. The use of prior knowledge and multiple images opens new ways to reconstruct surfaces and scenes impossible or error prone with previous methods. Another advantage is improved occlusion handling. Pixels that are partly occluded contribute to the reconstruction results. The proposed shifting of some of the computation to graphics hardware (GPU) results in a significant speed improvement compared to pure CPU-based implementations.
Spatio-Temporal Registration Techniques for Relightable 3D Video

By jointly applying a model-based marker-less motion capture approach and multi-view texture generation 3D videos of human actors can be reconstructed from multi-view video streams. If the input data were recorded under calibrated lighting, the texture information can also be used to measure time-varying surface reflectance. This way, 3D videos can be realistically displayed under novel lighting conditions. Reflectance estimation is only feasible if the multi-view texture-to-surface registration is consistent over time. In this paper, we propose two image-based warping methods that compensate registration errors due to inaccurate model geometry and shifting of apparel over the body.
An Augmented Reality System for Astronomical Observations

Anyone who has gazed through the eyepiece of an astronomical telescope knows that, with the exception of the Moon and the planets, extra-solar astronomical objects are disappointing to observe visually. This is mainly due to their low surface brightness, but also depends on the visibility, sky brightness and telescope aperture. We propose a system which projects images of astronomical objects (with focus on nebulae and galaxies), animations and additional information directly into the eyepiece view of an astronomical telescope. As the telescope orientation is queried continuously, the projected image is adapted in real-time to the currently visible field of view. For projection, a custom-built video projection module with high contrast and low maximum luminance value was developed. With this technology visitors to public observatories have the option to experience the richness of faint astronomical objects while directly looking at them through a telescope.
Spacetime-coherent geometry reconstruction from multiple video streams

By reconstructing time-varying geometry one frame at a time, one ignores the continuity of natural motion, wasting useful information about the underlying video-image formation process and taking into account temporally discontinuous reconstruction results. In 4D spacetime, the surface of a dynamic object describes a continuous 3D hyper-surface. This hyper-surface can be implicitly defined as the minimum of an energy functional designed to optimize photo-consistency. Based on an Eider-Lagrange reformulation of the problem, we find this hyper-surface from a handful of synchronized video recordings. The resulting object geometry varies smoothly over time, and intermittently invisible object regions are correctly interpolated from previously and/or future frames.
Improving the visual analysis of high-dimensional datasets using quality measures

Modern visualization methods are needed to cope with very high-dimensional data. Efficient visual analytical techniques are required to extract the information content in these data. The large number of possible projections for each method, which usually grow quadrat-ically or even exponentially with the number of dimensions, urges the necessity to employ automatic reduction techniques, automatic sorting or selecting the projections, based on their information-bearing content. Different quality measures have been successfully applied for several specified user tasks and established visualization techniques, like Scatterplots, Scatterplot Matrices or Parallel Coordinates. Many other popular visualization techniques exist, but due to the structural differences, the measures are not directly applicable to them and new approaches are needed. In this paper we propose new quality measures for three popular visualization methods: Radviz, Pixel-Oriented Displays and Table Lenses. Our experiments show that these measures efficiently guide the visual analysis task.
The virtual video camera: Simplified 3DTV acquisition and processing

We present the Virtual Video Camera, and end-to-end image-based free viewpoint framework. Novel spatial and temporal views are rendered from a set of unsynchronized, uncalibrated video streams. These loose requirements make outdoor recordings with consumer grade camcorders possible. Sophisticated visual effects can be devised in post-production, and stereoscopic output videos can be rendered with a variable baseline. Our Virtual Video Camera integrates seamlessly into traditional 2D and 3D post-production workflows.
Motion Field Estimation from Alternate Exposure Images

Traditional optical flow algorithms rely on consecutive short-exposed images. In this work, we make use of an additional long-exposed image for motion field estimation. Long-exposed images integrate motion information directly in the form of motion-blur. With this additional information, more robust and accurate motion fields can be estimated. In addition, the moment of occlusion can be determined. Considering the basic signal-theoretical problem in motion field estimation, we exploit the fact that long-exposed images integrate motion information to prevent temporal aliasing. A suitable image formation model relates the long-exposed image to preceding and succeeding short-exposed images in terms of dense 2D motion and per-pixel occlusion/disocclusion timings. Based on our image formation model, we describe a practical variational algorithm to estimate the motion field not only for visible image regions but also for regions getting occluded. Results for synthetic as well as real-world scenes demonstrate the validity of the approach.
Monocular albedo reconstruction

Reconstructing objects from monocular input video data is a challenging task. Since cues about shading and illumination are ambiguous from a single viewpoint, simultaneously distinguishing between material properties, texture, and illumination is infeasible. To properly reconstruct scene illumination a model of the object surface is mandatory. As most 3D surface reconstruction, surface refinement, and illumination reconstruction techniques rely on a diffuse object surface model, we propose a technique to robustly reconstruct surface albedo from monocular input video data and a 3D mesh representation of an object. For all frames of the video sequence we collect color samples for visible vertices under varying illumination directions and correct for possible ambient occlusion using mesh geometry information. We then use a two-staged clustering approach to recover the most plausible surface albedo for every mesh vertex. The presented approach does not require any a-priori illumination model and is evaluated using synthetic as well as real-world test sequences.
Space-time isosurface evolution for temporally coherent 3D reconstruction

We model the dynamic geometry of a time-varying scene as a 3D isosurface in space-time. The intersection of the isosurface with planes of constant time yields the geometry at a single time instant. An optimal fit of our model to multiple video sequences is defined as the minimum of an energy functional. This functional is given by an integral over the entire hypersurface, which is designed to optimize photo-consistency. A PDE-based evolution derived from the Euler-Lagrange equation maximizes consistency with all of the given video data simultaneously. The result is a 3D model of the scene which varies smoothly over time. The geometry reconstructed by this scheme is significantly better than results obtained by space-carving approaches that do not enforce temporal coherence.
RGB-guided depth map compression via Compressed Sensing and Sparse Coding

We present a novel approach for compression of depth maps based on Compressed Sensing and Sparse Coding. Our proposed scheme compresses and stores the depth map, and then-during the decompression step-makes use of the readily available additional RGB information to guide the reconstruction. We introduce additional constraints to the underlying optimization problem enforcing the correctness of the RGB image in the decompression step. A comparison with established compression schemes shows that our proposed method leads to a lower error rate at high compression ratios.
The Good, the Bad and the Ugly: Attractive Portraits from Video Sequences

Taking a portrait picture is frequently an uncomfortable task for the photographed person, and the resulting shots are in many cases unnatural and unsatisfactory. A common solution is to take a large number of pictures and then select the favorite ones. Instead of a manual selection, an automatic selection of good portraits from a captured video sequence can increase the chances to obtain a pleasing portrait picture without much effort. In this paper, we present an automatic solution for classification of face images from video sequences. This, in turn, can not only ease the task of taking good portrait pictures, but it can also make the acquisition of portraits more comfortable. (4 pages)
Visualization of Astronomical Nebulae via Distributed Multi-GPU Compressed Sensing Tomography

The 3D visualization of astronomical nebulae is a challenging problem since only a single 2D projection is observable from our fixed vantage point on Earth. We attempt to generate plausible and realistic looking volumetric visualizations via a tomographic approach that exploits the spherical or axial symmetry prevalent in some relevant types of nebulae. Different types of symmetry can be implemented by using different randomized distributions of virtual cameras. Our approach is based on an iterative compressed sensing reconstruction algorithm that we extend with support for position-dependent volumetric regularization and linear equality constraints. We present a distributed multi-GPU implementation that is capable of reconstructing high-resolution datasets from arbitrary projections. Its robustness and scalability are demonstrated for astronomical imagery from the Hubble Space Telescope. The resulting volumetric data is visualized using direct volume rendering. Compared to previous approaches, our method preserves a much higher amount of detail and visual variety in the 3D visualization, especially for objects with only approximate symmetry.
Interactive rendering of translucent objects

This paper presents a rendering method for translucent objects, in which view point and illumination can be modified at interactive rates. In a preprocessing step the impulse response to incoming light impinging at each surface point is computed and stored in two different ways: The local effect on close-by surface points is modeled as a per-texel filter kernel that is applied to a texture map representing the incident illumination. The global response (i.e. light shining through the object) is stored as vertex-to-vertex throughput factors for the triangle mesh of the object. During rendering, the illumination map for the object is computed according to the current lighting situation and then filtered by the precomputed kernels. The illumination map is also used to derive the incident illumination on the vertices which is distributed via the vertex-to-vertex throughput factors to the other vertices. The final image is obtained by combining the local and global response. We demonstrate the performance of our method for several models.
ElectroEncephaloGraphics: Making Waves in Computer Graphics Research

Electroencephalography (EEG) is a novel modality for investigating perceptual graphics problems. Until recently, EEG has predominantly been used for clinical diagnosis, in psychology, and by the brain-computer-interface community. Researchers are extending it to help understand the perception of visual output from graphics applications and to create approaches based on direct neural feedback. Researchers have applied EEG to graphics to determine perceived image and video quality by detecting typical rendering artifacts, to evaluate visualization effectiveness by calculating the cognitive load, and to automatically optimize rendering parameters for images and videos on the basis of implicit neural feedback.
Synthetic Generation of High-Dimensional Datasets

Generation of synthetic datasets is a common practice in many research areas. Such data is often generated to meet specific needs or certain conditions that may not be easily found in the original, real data. The nature of the data varies according to the application area and includes text, graphs, social or weather data, among many others. The common process to create such synthetic datasets is to implement small scripts or programs, restricted to small problems or to a specific application. In this paper we propose a framework designed to generate high dimensional datasets. Users can interactively create and navigate through multi dimensional datasets using a suitable graphical user-interface. The data creation is driven by statistical distributions based on a few user-defined parameters. First, a grounding dataset is created according to given inputs, and then structures and trends are included in selected dimensions and orthogonal projection planes. Furthermore, our framework supports the creation of complex non-orthogonal trends and classified datasets. It can successfully be used to create synthetic datasets simulating important trends as multidimensional clusters, correlations and outliers.
Reconstructing the geometry of flowing water

We present a recording scheme, image formation model and reconstruction method that enables image-based modeling of flowing bodies of water from multivideo input data. The recorded water is dyed with a fluorescent chemical to measure the thickness of a column of water which leads to an image formation model based on integrated emissivities along a viewing ray. This model allows for a photo-consistency based error measure for a weighted minimal surface, which is recovered using a PDE obtained from the Euler-Lagrangian formulation of the problem. The resulting equation is solved using the level set method
Video Colorization Using Parallel Optimization in Feature Space

We present a new scheme for video colorization using optimization in rotation-aware Gabor feature space. Most current methods of video colorization incur temporal artifacts and prohibitive processing costs, while this approach is designed in a spatiotemporal manner to preserve temporal coherence. The parallel implementation on graphics hardware is also facilitated to achieve realtime performance of color optimization. By adaptively clustering video frames and extending Gabor filtering to optical flow computation, we can achieve real-time color propagation within and between frames. Temporal coherence is further refined through user scribbles in video frames. The experimental results demonstrate that our proposed approach is efficient in producing high-quality colorized videos.
Temporal Video Filtering and Exposure Control for Perceptual Motion Blur

We propose the computation of a perceptual motion blur in videos. Our technique takes the predicted eye motion into account when watching the video. Compared to traditional motion blur recorded by a video camera our approach results in a perceptual blur that is closer to reality. This postprocess can also be used to simulate different shutter effects or for other artistic purposes. It handles real and artificial video input, is easy to compute and has a low additional cost for rendered content. We illustrate its advantages in a user study using eye tracking.
A Fast and Robust Approach to Lane Marking Detection and Lane Tracking

We present a lane detection algorithm that robustly detects and tracks various lane markings in real-time. The first part is a feature detection algorithm that transforms several input images into a top view perspective and analyzes local histograms. For this part we make use of state-of-the-art graphics hardware. The second part fits a very simple and flexible lane model to these lane marking features. The algorithm was thoroughly tested on an autonomous vehicle that was one of the finalists in the 2007 DARPA Urban Challenge. In combination with other sensors, i.e. a lidar, radar and vision based obstacle detection and surface classification, the autonomous vehicle is able to drive in an urban scenario at up to 15 mp/h.