OpenRaiser/Intern-Atlas huggingface dataset:
Efficiently selecting regions for scene understanding
Authors: M. Pawan Kumar, Daphne Koller
Venue: Conference CVPR 2010
Year: 2010

Recent advances in scene understanding and related tasks have highlighted the importance of using regions to reason about high-level scene structure. Typically, the regions are selected beforehand and then an energy function is defined over them. This two step process suffers from the following deficiencies: (i) the regions may not match the boundaries of the scene entities, thereby introducing errors; and (ii) as the regions are obtained without any knowledge of the energy function, they may not be suitable for the task at hand. We address these problems by designing an efficient approach for obtaining the best set of regions in terms of the energy function itself. Each iteration of our algorithm selects regions from a large dictionary by solving an accurate linear programming relaxation via dual decomposition. The dictionary of regions is constructed by merging and intersecting segments obtained from multiple bottom-up over-segmentations. To demonstrate the usefulness of our algorithm, we consider the task of scene segmentation and show significant improvements over state of the art methods.
Combining appearance models and markov random fields for category level object segmentation
Authors: Diane Larlus, Frédéric Jurie
Venue: Conference CVPR 2008
Year: 2008

Object models based on bag-of-words representations can achieve state-of-the-art performance for image classification and object localization tasks. However, as they consider objects as loose collections of local patches they fail to accurately locate object boundaries and are not able to produce accurate object segmentation. On the other hand, Markov random field models used for image segmentation focus on object boundaries but can hardly use the global constraints necessary to deal with object categories whose appearance may vary significantly. In this paper we combine the advantages of both approaches. First, a mechanism based on local regions allows object detection using visual word occurrences and produces a rough image segmentation. Then, a MRF component gives clean boundaries and enforces label consistency, guided by local image cues (color, texture and edge cues) and by long-distance dependencies. Gibbs sampling is used to infer the model. The proposed method successfully segments object categories with highly varying appearances in the presence of cluttered backgrounds and large view point changes. We show that it outperforms published results on the Pascal VOC 2007 dataset.
Shape of Gaussians as Feature Descriptors
Authors: Liyu Gong, Tianjiang Wang, Fang Liu 0011
Venue: Conference CVPR 2009
Year: 2009

This paper introduces a feature descriptor called Shape of Gaussian (SOG), which is based on a general feature descriptor design framework called Shape of Signal Probability Density Function (SOSPDF). SOSPDF takes the shape of a signal's probability density function (pdf) as its feature. Under such a view, both histogram and region covariance often used in computer vision are SOSPDF features. Histogram describes SOSPDF by a discrete approximation way. Region covariance describes SOSPDF as an incomplete parameterized multivariate Gaussian distribution. Our proposed SOG descriptor is a full parameterized Gaussian, so it has all the advantages of region covariance and is more effective. Furthermore, we identify that SOGs form a Lie group. Based on Lie group theory, we propose a distance metric for SOG. We test SOG features in tracking problem. Experiments show better tracking results compared with region covariance. Moreover, experiment results indicate that SOG features attempt to harvest more useful information and are less sensitive against noise.
Cache-efficient Graph Cuts on Structured Grids
Authors: Ondrej Jamriska, Daniel Sýkora, Alexander Hornung
Venue: Conference CVPR 2012
Year: 2012

Finding minimal cuts on graphs with a grid-like structure has become a core task for solving many computer vision and graphics related problems. However, computation speed and memory consumption oftentimes limit the effective use in applications requiring high resolution grids or interactive response. In particular, memory bandwidth represents one of the major bottlenecks even in today's most efficient implementations. We propose a compact data structure with cache-efficient memory layout for the representation of graph instances that are based on regular N-D grids with topologically identical neighborhood systems. For this common class of graphs our data structure allows for 3 to 12 times higher grid resolutions and a 3- to 9-fold speedup compared to existing approaches. Our design is agnostic to the underlying algorithm, and hence orthogonal to other optimizations such as parallel and hierarchical processing. We evaluate the performance gain on a variety of typical problems including 2D/3D segmentation, colorization, and stereo. All experiments show an unconditional improvement in terms of speed and memory consumption, with graceful performance degradation for graphs with increasing topological irregularities.
Parallel Graph-cuts by Adaptive Bottom-up Merging
Authors: Jiangyu Liu, Jian Sun 0001
Venue: Conference CVPR 2010
Year: 2010

Graph-cuts optimization is prevalent in vision and graphics problems. It is thus of great practical importance to parallelize the graph-cuts optimization using today's ubiquitous multi-core machines. However, the current best serial algorithm by Boykov and Kolmogorov (called the BK algorithm) still has the superior empirical performance. It is non-trivial to parallelize as expensive synchronization overhead easily offsets the advantage of parallelism. In this paper, we propose a novel adaptive bottom-up approach to parallelize the BK algorithm. We first uniformly partition the graph into a number of regularly-shaped disjoint subgraphs and process them in parallel, then we incrementally merge the subgraphs in an adaptive way to obtain the global optimum. The new algorithm has three benefits: 1) it is more cache-friendly within smaller subgraphs; 2) it keeps balanced workloads among computing cores; 3) it causes little overhead and is adaptable to the number of available cores. Extensive experiments in common applications such as 2D/3D image segmentations and 3D surface fitting demonstrate the effectiveness of our approach.
I know what you did last summer: object-level auto-annotation of holiday snaps
Authors: Stephan Gammeter, Lukas Bossard, Till Quack, Luc Van Gool
Venue: Conference ICCV 2009
Year: 2009

The state-of-the art in visual object retrieval from large databases allows to search millions of images on the object level. Recently, complementary works have proposed systems to crawl large object databases from community photo collections on the Internet. We combine these two lines of work to a large-scale system for auto-annotation of holiday snaps. The resulting method allows for automatic labeling objects such as landmark buildings, scenes, pieces of art etc. at the object level in a fully automatic manner. The labeling is multi-modal and consists of textual tags, geographic location, and related content on the Internet. Furthermore, the efficiency of the retrieval process is optimized by creating more compact and precise indices for visual vocabularies using background information obtained in the crawling stage of the system. We demonstrate the scalability and precision of the proposed method by conducting experiments on millions of images downloaded from community photo collections on the Internet.
Face illumination transfer through edge-preserving filters
Authors: Xiaowu Chen 0001, Mengmeng Chen, Xin Jin 0015, Qinping Zhao
Venue: Conference CVPR 2011
Year: 2011

This article proposes a novel image-based method to transfer illumination from a reference face image to a target face image through edge-preserving filters. According to our method, only a single reference image, without any knowledge of the 3D geometry or material information of the target face, is needed. We first decompose the lightness layers of the reference and the target images into large-scale and detail layers through weighted least square (WLS) filter after face alignment. The large-scale layer of the reference image is filtered with the guidance of the target image. Adaptive parameter selection schemes for the edge-preserving filters is proposed in the above two filtering steps. The final relit result is obtained by replacing the large-scale layer of the target image with that of the reference image. We acquire convincing relit result on numerous target and reference face images with different lighting effects and genders. Comparisons with previous work show that our method is less affected by geometry differences and can preserve better the identification structure and skin color of the target face.
Harmony potentials for joint classification and segmentation
Authors: Josep M. Gonfaus, Xavier Boix Bosch, Joost van de Weijer 0001, Andrew D. Bagdanov, Joan Serrat Gual, Jordi Gonzàlez 0001
Venue: Conference CVPR 2010
Year: 2010

Hierarchical conditional random fields have been successfully applied to object segmentation. One reason is their ability to incorporate contextual information at different scales. However, these models do not allow multiple labels to be assigned to a single node. At higher scales in the image, this yields an oversimplified model, since multiple classes can be reasonable expected to appear within one region. This simplified model especially limits the impact that observations at larger scales may have on the CRF model. Neglecting the information at larger scales is undesirable since class-label estimates based on these scales are more reliable than at smaller, noisier scales. To address this problem, we propose a new potential, called harmony potential, which can encode any possible combination of class labels. We propose an effective sampling strategy that renders tractable the underlying optimization problem. Results show that our approach obtains state-of-the-art results on two challenging datasets: Pascal VOC 2009 and MSRC-21.
Depth from optical turbulence
Authors: Yuandong Tian, Srinivasa G. Narasimhan, Alan Van Nevel
Venue: Conference CVPR 2012
Year: 2012

Turbulence near hot surfaces such as desert terrains and roads during the summer, causes shimmering, distortion and blurring in images. While recent works have focused on image restoration, this paper explores what information about the scene can be extracted from the distortion caused by turbulence. Based on the physical model of wave propagation, we first study the relationship between the scene depth and the amount of distortion caused by homogenous turbulence. We then extend this relationship to more practical scenarios such as finite extent and height-varying turbulence, and present simple algorithms to estimate depth ordering, depth discontinuity and relative depth, from a sequence of short exposure images. In the case of general non-homogenous turbulence, we show that a statistical property of turbulence can be used to improve long-range structure-from-motion (or stereo). We demonstrate the accuracy of our methods in both laboratory and outdoor settings and conclude that turbulence (when present) can be a strong and useful depth cue.
Fundamental matrix for cameras with radial distortion
Authors: João Pedro Barreto 0001, Kostas Daniilidis
Venue: Conference ICCV 2005
Year: 2005

When deploying a heterogeneous camera network or when we use cheap zoom cameras like in cell-phones, it is not practical, if not impossible to off-line calibrate the radial distortion of each camera using reference objects. It is rather desirable to have an automatic procedure without strong assumptions about the scene. In this paper, we present a new algorithm for estimating the epipolar geometry of two views where the two views can be radially distorted with different distortion factors. It is the first algorithm in the literature solving the case of different distortion in the left and right view linearly and without assuming the existence of lines in the scene. Points in the projective plane are lifted to a quadric in three-dimensional projective space. A radial distortion of the projective plane results to a matrix transformation in the space of lifted coordinates. The new epipolar constraint depends linearly on a 4 /spl times/ 4 radial fundamental matrix which has 9 degrees of freedom. A complete algorithm is presented and tested on real imagery.
Augmented Reality System for Surgical Navigation using Robust Target Vision
Authors: Akio Kosaka, Akito Saito, Yukihito Furuhashi, Takao Shibasaki
Venue: Conference CVPR 2000
Year: 2000

This paper presents a robust and accurate vision-based augmented reality system for surgical navigation. The key point of our system is a robust and real-time monocular vision algorithm to estimate the 3D pose of surgical tools, utilizing specially designed code markers and Kalman filter-based position updating. The vision system is not impaired by occlusion and rapid change of illumination. The augmented reality system superimposes the 3D object wireframe onto the live viewing image taken from the surgical microscope as well as displaying other useful navigation information, while allowing the surgeons to freely change its zoom and focus for viewing. The experimental results verified the robustness and usefulness of the system, and acquired the image registration error less than 2 mm.
An effective document image deblurring algorithm
Authors: Xiaogang Chen, Xiangjian He, Jie Yang 0002, Qiang Wu 0001
Venue: Conference CVPR 2011
Year: 2011

Deblurring camera-based document image is an important task in digital document processing, since it can improve both the accuracy of optical character recognition systems and the visual quality of document images. Traditional deblurring algorithms have been proposed to work for natural-scene images. However the natural-scene images are not consistent with document images. In this paper, the distinct characteristics of document images are investigated. We propose a content-aware prior for document image deblurring. It is based on document image foreground segmentation. Besides, an upper-bound constraint combined with total variation based method is proposed to suppress the rings in the deblurred image. Comparing with the traditional general purpose deblurring methods, the proposed deblurring algorithm can produce more pleasing results on document images. Encouraging experimental results demonstrate the efficacy of the proposed method.
Salient object detection for searched web images via global saliency
Authors: Peng Wang 0001, Jingdong Wang 0001, Gang Zeng, Jie Feng 0012, Hongbin Zha, Shipeng Li 0001
Venue: Conference CVPR 2012
Year: 2012

In this paper, we deal with the problem of detecting the existence and the location of salient objects for thumbnail images on which most search engines usually perform visual analysis in order to handle web-scale images. Different from previous techniques, such as sliding window-based or segmentation-based schemes for detecting salient objects, we propose to use a learning approach, random forest in our solution. Our algorithm exploits global features from multiple saliency indicators to directly predict the existence and the position of the salient object. To validate our algorithm, we constructed a large image database collected from Bing image search, that contains hundreds of thousands of manually labeled web images. The experimental results using this new database and the resized MSRA database [16] demonstrate that our algorithm outperforms previous state-of-the-art methods.
A rational function lens distortion model for general cameras
Authors: David Claus, Andrew W. Fitzgibbon
Venue: Conference CVPR 2005
Year: 2005

We introduce a new rational function (RF) model for radial lens distortion in wide-angle and catadioptric lenses, which allows the simultaneous linear estimation of motion and lens geometry from two uncalibrated views of a 3D scene. In contrast to existing models which admit such linear estimates, the new model is not specialized to any particular lens geometry, but is sufficiently general to model a variety of extreme distortions. The key step is to define the mapping between image (pixel) coordinates and 3D rays in camera coordinates as a linear combination of nonlinear functions of the image coordinates. Like a "kernel trick", this allows a linear algorithm to estimate nonlinear models, and in particular offers a simple solution to the estimation of nonlinear image distortion. The model also yields an explicit form for the epipolar curves, allowing correspondence search to be efficiently guided by the epipolar geometry. We show results of an implementation of the RF model in estimating the geometry of a real camera lens from uncalibrated footage, and compare the estimate to one obtained using a calibration grid.
Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation
Authors: Sudipta N. Sinha, Marc Pollefeys
Venue: Conference ICCV 2005
Year: 2005

This paper describes a novel approach for reconstructing a closed continuous surface of an object from multiple calibrated color images and silhouettes. Any accurate reconstruction must satisfy (1) photo-consistency and (2) silhouette consistency constraints. Most existing techniques treat these cues identically in optimization frameworks where silhouette constraints are traded off against photo-consistency and smoothness priors. Our approach strictly enforces silhouette constraints, while optimizing photo-consistency and smoothness in a global graph-cut framework. We transform the reconstruction problem into computing max-flow/min-cut in a geometric graph, where any cut corresponds to a surface satisfying exact silhouette constraints (its silhouettes should exactly coincide with those of the visual hull); a minimum cut is the most photo-consistent surface amongst them. Our graph-cut formulation is based on the rim mesh, (the combinatorial arrangement of rims or contour generators from many views) which can be computed directly from the silhouettes. Unlike other methods, our approach enforces silhouette constraints without introducing a bias near the visual hull boundary and also recovers the rim curves. Results are presented for synthetic and real datasets.
Synthetic view augmentation for sign language recognition
Authors: Yuting Peng, Peiqi Jiao, Honggang Zou, Yuecong Min, Xilin Chen
Venue: Conference WWW 2025
Year: 2025

Sign language, as a visual language, is inherently spatial and dynamic, characterized by various hand shapes, movements, facial expressions, and body postures. In real-world scenarios, sign language recognition (SLR) systems are deployed in diverse environments with varying lighting conditions, backgrounds, and user positions, and models trained on data from a single viewpoint often struggle to generalize effectively to new perspectives. To address these limitations, this paper focuses on cross-view SLR and proposes a synthetic view augmentation pipeline to facilitate SLR models learning viewpoint-invariant features during training, thereby improving their performance on unseen viewpoints. Additionally, we introduce the XV-SLR model, which incorporates multi-modality information further to enhance the robustness of cross-view sign language recognition. The proposed method achieves a top-1 accuracy of 56.87% under the RGB setting and 57.97% under the RGB-D setting and outperforms other approaches in the Cross-View Isolated Sign Language Recognition Challenge (CV-ISLR). Code is available at https://github.com/VIPL-SLP/XV-SLR.
Auslanweb: A scalable web-based australian sign language communication system for deaf and hearing individuals
Authors: Xin Shen, Heming Du, Hongwei Sheng, Lincheng Li, Kaihao Zhang
Venue: Conference WWW 2025
Year: 2025

Effective communication between the deaf community and hearing individuals facilitates social inclusion, equal opportunities, and the dignity of vulnerable populations. However, existing region-specific sign language systems are constrained by limited training datasets and narrow topic domains, rendering them ineffective for bridging the linguistic gaps between sign languages and spoken languages. Auslan, as the sign language specific to Australia, still lacks a reliable bidirectional translation tool for effective communication. To address these challenges, we propose AuslanWeb, a web-based system for bidirectional translation of both isolated and successive sign language. For the former, AuslanWeb achieves high-precision mapping between isolated signs (glosses) and spoken language words or phrases through a multimodal recognition system and a versatile Auslan dictionary. For the latter, it leverages the advanced contextual understanding and text generation capabilities of Large Language Models (LLMs) to support bidirectional translation between successive sign language videos and long-form spoken language. By integrating linguistic structure with advanced AI capabilities, AuslanWeb overcomes the limitations of dataset dependency and enhances the scalability of sign language translation systems. The effectiveness of the system is further validated through user feedback, receiving consistent praise from Auslan experts, Australian deaf individuals, and volunteers. The demo video of AuslanWeb is provided here.
Cross-view isolated sign language recognition challenge: Design, results and future research
Authors: Xin Shen, Heming Du, Miao Xu, Miaomiao Liu, Xin Yu
Venue: Conference WWW 2025
Year: 2025

The performance of Isolated Sign Language Recognition (ISLR) systems has improved considerably in recent years. However, existing ISLR models often struggle to generalize across different viewpoints, limiting their practical usability in real-world applications. To highlight this limitation, we introduce the Cross-View ISLR Challenge (CV-ISLR), a workshop focused on improving cross-view generalization in ISLR task. Our challenge fosters research on robust ISLR models capable of recognizing signs from various viewpoints. It also encourages the development of novel multi-modal learning techniques and helps bridge the gap between academic research and real-world deployment. We present an overview of the CV-ISLR Challenge, organized as part of the ACM Web Conference 2025 (WWW 2025). Participants are evaluated on MM-WLAuslan, a newly introduced large-scale multi-modal Australian Sign Language (Auslan) dataset. It contains 3,215 sign glosses and over 282K isolated sign video samples, performed by 73 different signers and recorded from three distinct RGB-D camera viewpoints. The challenge attracted 23 participants on the RGB track and 11 on the RGB+Depth track, leading to more than 120 submissions in total. We analyze and discuss the challenge design, top-performing solutions, and insights for future CV-ISLR research. For more details about the challenge, please visit our official website.