nils-herrmann/clean_scirepeval huggingface dataset:
Sparse recursive cost aggregation towards O(1) complexity local stereo matching

The complexity of the local stereo matching methods mainly increases with disparity search range and cost aggregation step. Joint elimination of the those complexity factors is a challenging task as a consequence of the contradicting nature of the methods attacking the reduction on the complexity factors. In this paper, that challenge is addressed and for the disparity search range reducing approaches, an efficient cost aggregation method is proposed by reformulating the filtering scheme of the recursive edge-aware filters which have been proved to be efficient approaches for cost aggregation. The proposed method is exploited by a hierarchical stereo matching approach. In that manner, fixed number of disparity candidates are tested for each pixel, regardless of the search space and the cost aggregation for each candidate is performed with constant complexity. The experimental results validate that the proposed approach has linear complexity with the image size and show that in practice it speeds up the recursive approaches almost four times with 0.01-0.96% decrease in matching accuracy. Compared to the state-of-the-art techniques, the proposed method is possibly the fastest approach with a competitive accuracy based on Middlebury benchmarking.
Gaussian Mixture Models for Temporal Depth Fusion

Sensing the 3D environment of a moving robot is essential for collision avoidance. Most 3D sensors produce dense depth maps, which are subject to imperfections due to various environmental factors. Temporal fusion of depth maps is crucial to overcome those. Temporal fusion is traditionally done in 3D space with voxel data structures, but it can be approached by temporal fusion in image space, with potential benefits in reduced memory and computational cost for applications like reactive collision avoidance for micro air vehicles. In this paper, we present an efficient Gaussian Mixture Models based depth map fusion approach, introducing an online update scheme for dense representations. The environment is modeled from an ego-centric point of view, where each pixel is represented by a mixture of Gaussian inverse-depth models. Consecutive frames are related to each other by transformations obtained from visual odometry. This approach achieves better accuracy than alternative image space depth map fusion techniques at lower computational cost.
LASP: Local adaptive super-pixels

In this study, a novel gradient ascent approach is proposed for super-pixel extraction in which spectral statistics and super-pixel geometry are utilized to obtain an optimal Bayesian classifier for pixel to super-pixel label assignment. Utilization of the spectral variances and super-pixel areas reduces the dependency on user selected global parameters, while increasing robustness and adaptability. Proposed Local Adaptive Super-Pixels (LASP) approach exploits hexagonal tiling, while achieving some refinement during initialization in order to improve computation time and accuracy. The experiments conducted on Berkeley segmentation database show that LASP outperforms the existing methods in terms of boundary recall and computation time. Moreover, the proposed method provides lower bleeding error performance compared to the existing gradient ascent techniques.
Super pixel extraction via convexity induced boundary adaptation

This study presents an efficient super-pixel extraction algorithm with major contributions to the state-of-the-art in terms of accuracy and computational complexity. Segmentation accuracy is improved through convexity constrained geodesic distance utilization; while computational efficiency is achieved by replacing complete region processing with boundary adaptation idea. Starting from the uniformly distributed rectangular equal-sized super-pixels, region boundaries are adapted to intensity edges iteratively by assigning boundary pixels to the most similar neighboring super-pixels. At each iteration, super-pixel regions are updated and hence progressively converging to compact pixel groups. Experimental results with state-of-the-art comparisons, validate the performance of the proposed technique in terms of both accuracy and speed.
Watermarking of Free-view Video

With the advances in image based rendering (IBR) in recent years, generation of a realistic arbitrary view of a scene from a number of original views has become cheaper and faster. One of the main applications of this progress has emerged as free-view TV(FTV), where TV-viewers select freely the viewing position and angle via IBR on the transmitted multiview video. Noting that the TV-viewer might record a personal video for this arbitrarily selected view and misuse this content, it is apparent that copyright and copy protection problems also exist and should be solved for FTV. In this paper, we focus on this newly emerged problem by proposing a watermarking method for free-view video. The watermark is embedded into every frame of multiple views by exploiting the spatial masking properties of the human visual system. Assuming that the position and rotation of the virtual camera is known, the proposed method extracts the watermark successfully from an arbitrarily generated virtual image. In order to extend the method for the case of an unknown virtual camera position and rotation, the transformations on the watermark pattern due to image based rendering operations are analyzed. Based upon this analysis, camera position and homography estimation methods are proposed for the virtual camera. The encouraging simulation results promise not only a novel method, but also a new direction for watermarking research.
Reduction of crosstalk in stereoscopic displays

This paper proposes a method for reducing crosstalk between right and left images displayed on a stereoscopic display. The underlying reason of crosstalk in stereoscopic displays is mainly due to the technological methodology applied in the display hardware. Lack of stereoscopic isolation between the left and right image pairs causes both images to be partly observed by each eye. This causes deterioration or even total loss in the observed 3D perception. The proposed study offers a method to reduce the observed crosstalk in real time and hence creating a superior 3D perceptual quality.
Low cost FPGA design and implementation of a stereo matching system for 3D-TV applications

There exist numerous solutions to the stereo matching problem and many have been implemented on FPGAs. However, these solutions are conceived and designed as a stand-alone module without considering the area constraints and hard deadlines of the underlying application. In this paper, we propose a low-cost and real-time stereo matching system that is specifically designed to be integrated into the video pipeline of a Full-HD 3D-TV system, which is compliant with HDMI 1.4a specification. The proposed hardware architecture was implemented in VHDL and mapped on low-cost Spartan6-XC6SLX9 FPGA. The implementation runs at 148.5 MHz and achieves 60 fps in order to meet the HDMI 1.4a requirements, while using only 3k LUTs and 29 of 16Kb-BRAMs. Compared to existing work, the proposed implementation can generate high quality disparity maps with a much compact implementation. Moreover, to the best of our best knowledge, we present the first hardware implementation of information permeability based stereo matching algorithm, which exploits one of the most efficient aggregation methodologies compared to the state-of-the-art. We also identify the problems brought out during the system level integration and provide novel solutions to them.
Segment-Based Stereo-Matching Via Plane and Angle Sweeping

A novel approach for segment-based stereo matching problem is presented, based on a modified plane-sweeping strategy. The space is initially divided into planes that are located at different depth levels via plane sweeping by the help of region-wise planarity assumption for the scene. Over-segmented homogenous color regions are utilized for defining planar segment boundaries and plane equations are determined by angle sweeping at different planes. The robustness of depth map estimates is improved by warping segments into the other image via the resulting homographies. In order to refine the reconstruction quality and update segment depths, as well as plane normals, with smoothness and visibility constraints, a greedy search algorithm is applied. Based on the simulation results, the proposed algorithm handles large un-textured regions, depth discontinuities at object boundaries and slanted surfaces. Moreover, the algorithm could be easily upgraded from stereo to multi-view case, since 3D plane equations are already determined.
Region-Based Dense Depth Extraction from Multi-View Video

A novel multi-view region-based dense depth map estimation problem is presented, based on a modified plane-sweeping strategy. In this approach, the whole scene is assumed to be region-wise planar. These planar regions are defined by back-projections of the over-segmented homogenous color regions on the images and the plane parameters are determined by angle-sweeping at different depth levels. The position and rotation of the plane patches are estimated robustly by minimizing a segment-based cost function, which considers occlusions, as well. The quality of depth map estimates is measured via reconstruction quality of the conjugate views, after warping segments into these views by the resulting homographies. Finally, a greedy-search algorithm is applied to refine the reconstruction quality and update the plane equations with visibility constraint. Based on the simulation results, it is observed that the proposed algorithm handles large un-textured regions, depth discontinuities at object boundaries, slanted surfaces, as well as occlusions.
Real-time stereo matching algorithm for 3DTV

A computationally efficient stereo matching algorithm is introduced providing high precision dense disparity maps via local aggregation approach. The proposed algorithm exploits a novel paradigm, namely separable successive weighted summation (SWS) among horizontal and vertical directions with constant operational complexity, providing effective connected 2D support regions based on local color similarities. The intensity adaptive aggregation enables crisp disparity maps which preserve object boundaries and depth discontinuities. The same procedure is also utilized to diffuse information through overlapped pixels during occlusion handling. According to the experimental results on Middlebury online stereo benchmark, the proposed method is one of the most effective local stereo algorithm providing high quality disparity models by unifying constant time filtering and weighted aggregation. Hence, the proposed algorithm provides a competitive alternative, with its efficient GPU and FPGA implementations, for various local methods in terms of achieving precise disparity maps from stereo video within fast execution time.
Asymmetric depth estimation for multiview

A novel multi-view depth map estimation algorithm is proposed with low computational complexity and high accuracy, while meeting requirements of N-view plus N-depth representation. The algorithm exploits an aggregation methodology, namely permeability filtering (PF), through computationally efficient two-pass aggregation producing adaptively weighted and connected 2D support regions for each pixel. This aggregation approach is quite efficient compared to its alternatives in the literature. The advantage of being occlusion free in multiple viewpoints results with high quality maps compared to stereo matching. Moreover, the asymmetric approach further increases efficiency without any loss in the precision, in terms of mapped reliable center depth map to neighboring cameras in order to avoid redundant computation. Hence, the proposed algorithm is shown as to be a competitive alternative via simulations to some leading state-of-the-art methods by reduced number of computations and efficient support calculation for extracting high quality dense depth maps from multiple camera views.
Temporally consistent dense depth map estimation via Belief Propagation

A method for estimating temporally and spatially consistent dense depth maps in multiple camera setups is presented which is important for reduction of perception artifacts in 3D displays. For this purpose, initially, depth estimation is performed for each camera with the piece-wise planarity assumption and Markov Random Field (MRF) based relaxation at each time instant independently. During the relaxation step, the consistency of depth maps for different cameras is also considered for the reliability of the models. Next, temporal consistency of the depth maps is achieved in two steps. In the first step, median filtering is applied for the static or background pixels, whose intensity levels are constant in time. Such an approach decreases the number of inconsistent depth values significantly. The second step considers the moving pixels and MRF formulation is updated by the additional information from the depth maps of the consequent frames through motion compensation. For the solution of the MRF formulation for both spatial and temporal consistency, Belief Propagation approach is utilized. The experiments indicate that the proposed method provide reliable dense depth map estimates both in spatial and temporal domains.
Region-based image segmentation via graph cuts

A graph theoretic color image segmentation algorithm is proposed, in which the popular normalized cuts image segmentation method is improved with modifications on its graph structure. The image is represented by a weighted undirected graph, whose nodes correspond to over-segmented regions, instead of pixels, that decreases the complexity of the overall algorithm. In addition, the link weights between the nodes are calculated through the intensity similarities of the neighboring regions. The irregular distribution of the nodes, as a result of such a modification, causes a bias towards combining regions with high number of links. This bias is removed by limiting the number of links for each node. Finally, segmentation is achieved by bipartitioning the graph recursively according to the minimization of the normalized cut measure. The simulation results indicate that the proposed segmentation scheme performs quite faster than the traditional normalized cut methods, as well as yielding better segmentation results due to its region-based representation.
Free-View Watermarking for Free-View Television

The recent advances in image based rendering (IBR) has pioneered a new technology, free-view television, in which TV-viewers select freely the viewing position and angle by the application of IBR on the transmitted multi-view video. Noting that the TV-viewer might also record a personal video for this arbitrarily selected view and misuse this content, it is apparent that copyright and copy protection problems also exist and should be solved for free-view TV. In this paper, we focus on this problem by proposing a watermarking method for free-view video. The watermark is embedded into every frame of multiple views by exploiting the spatial masking properties of the human visual system (HVS). Assuming that the position and rotation for the imagery view is known, the proposed method extracts the watermark successfully from an arbitrarily generated image. In order to extend the method for the case of an unknown imagery camera position and rotation, the modifications on the watermark pattern due to image based rendering operations are also analyzed. Based on this analysis, a camera position and homography estimation method is proposed considering the operations in image based rendering. The results show that the watermark detection is achieved successfully for the cases in which the imagery camera is arbitrarily located on the camera plane.
Recursive edge-aware filters for stereo matching

In this study, taxonomy of recursive edge-aware filters (REAF) is provided, with the introduction of new approaches to the state-of-the-art. The one tap recursive filters are classified according to recursion rate calculation, recursion type and the unification of reverse directions. In that manner, eight types of edge-aware recursive filters are defined, where only three of them are addressed in literature so far. Comprehensive analyses are provided based on computational complexity and filter characteristics which affect the use of such filters for various applications. In order to compare the capabilities of these filters, stereo matching, as one of the most common application area of edge-aware filters, is considered and extensive experiments are provided through well known datasets. The evaluation is conducted through large number of stereo pairs on both CPU and GPU with independent parameter optimization of each filter that provides fair comparison. According to the experimental results, advantages of un-normalized recursion for matching accuracy and sequential integration of reverse directions for execution speed are illustrated as important conclusions for future directions of REAFs.
Color calibration algorithm and FPGA implementation for stereo images

In this study, FPGA implementation of an algorithm for reducing color differences between left and right images, stemming from 3D video capturing problems such as different physical specifications, adjustments, and view angle of cameras capturing left and right video, and different direction of motion and are presented. Such problems that cannot be resolved during 3D video capturing, should be resolved before the video is presented to the end user (the viewer) for a comfortable viewing experience. The correction process includes, not only correcting the color gamut and contrast differences, but also the vertical and horizontal geometric shift corrections. In this study, after the algorithm is presented briefly, its hardware architecture and its implementation results on FPGA are presented in detail. Hardware implementation of the solution is realized on xc6slx45 FPGA from Xilinx Spartan6 family. The pre and post calibration images of stereo captured videos are given in second section of the paper.
Efficient graph-based image segmentation via speeded-up turbo pixels

An efficient graph based image segmentation algorithm exploiting a novel and fast turbo pixel extraction method is introduced. The images are modeled as weighted graphs whose nodes correspond to super pixels; and normalized cuts are utilized to obtain final segmentation. Utilizing super pixels provides an efficient and compact representation; the graph complexity decreases by hundreds in terms of node number. Connected K-means with convexity constraint is the key tool for the proposed super pixel extraction. Once the pixels are grouped into super pixels, iterative bi-partitioning of the weighted graph, as introduced in normalized cuts, is performed to obtain segmentation map. Supported by various experiments, the proposed two stage segmentation scheme can be considered to be one of the most efficient graph based segmentation algorithms providing high quality results.
Depth Assisted Object Segmentation in Multi-View Video

In this work, a novel and unified approach for multi-view video (MVV) object segmentation is presented. In the first stage, a region-based graph-theoretic color segmentation algorithm is proposed, in which the popular normalized cuts segmentation method is improved with some modifications on its graph structure. Segmentation is obtained by recursive bi-partitioning of a weighted graph of an initial over-segmentation mask. The available segmentation mask is also utilized during dense depth map estimation step, based on a novel modified plane- and angle- sweeping strategy for each of these regions. Dense depth estimation is achieved by region-wise planarity assumption for the whole scene, in which depth models are estimated for sub-regions. Finally, the multi-view image segmentation algorithm is extended to object segmentation in MVV by the additional optical flow information. The required motion field is obtained via region- based matching that has consistent parameterization with color segmentation and dense depth map estimation algorithms. Experimental results indicate that proposed approach segments semantically meaningful objects in MVV with high precision.
Sparse recursive filtering for O(1) stereo matching

Recursive edge-aware filters have been proved to be one of the most efficient approaches for cost aggregation in stereo matching. However, disparity search space dependency, as a result of full search, is the bottle-neck of these local techniques that prevent further reduction in computation. In this paper, the cost aggregation and correspondence search problems are re-formulated to enable adaptive search for each pixel during recursive operations that provides significant reduction in computational complexity. In that manner, fixed number of disparity candidates are tested for each pixel, regardless of the search space, that are aggregated through sparse recursive filtering. Hierarchical approach is exploited to pick disparity candidates for each pixel. The experimental results show that the proposed approach has linear complexity with the image size and in practice it speeds up the recursive approaches almost four times with a marginal decrease in matching accuracy. Compared to the state-of-the-art techniques, hierarchical sparse recursive aggregation is possibly the fastest approach with a competitive accuracy based on Middlebury benchmarking.
Image-Based Visual Perception and Representation for Collision Avoidance

We present a novel on-board perception system for collision avoidance by micro air vehicles (MAV). An egocentric cylindrical representation is utilized to model the world using forward-looking stereo vision. This efficient representation enables a 360° field of regard, as the vehicle moves around and disparity maps are fused temporally on the cylindrical map. For this purpose, we developed a new Gaussian Mixture Models-based disparity image fusion algorithm, with an extension to handle independently moving objects (IMO). The extension improves scene models in case of moving objects, where standard temporal fusion approaches cannot detect movers and introduce errors in world models due to the common static scene assumption. The on-board implementation of the vision pipeline provides disparity maps on a 360o egocentric cylindrical surface at 10 Hz. The perception output is used in our system by real-time motion planning with collision avoidance on the MAV.
Efficient super pixel extraction for image segmentation

An efficient graph based image segmentation algorithm exploiting a novel and fast turbo pixel extraction method is introduced. The images are modeled as weighted graphs whose nodes correspond to super pixels; and normalized cuts are utilized to obtain final segmentation. Utilizing super pixels provides an efficient and compact representation; the graph complexity decreases by hundreds in terms of node number. Connected K-means with convexity constraint is the key tool for the proposed super pixel extraction. Once the pixels are grouped into super pixels, iterative bi-partitioning of the weighted graph, as introduced in normalized cuts, is performed to obtain segmentation map. Supported by various experiments, the proposed two stage segmentation scheme can be considered to be one of the most efficient graph based segmentation algorithms providing high quality results.