Image analysis that produces an image-like array of symbolic or numerical elements (such as edge finding or depth map reconstruction) can be formulated as a labeling problem in which each element is to be assigned a label from a discrete or continuous label set. This formulation lends itself to algorithms, based on Bayesian probability theory, that support the combination of disparate sources of information, including prior knowledge. In the approach described here, local visual observations for each entity to be labeled (e.g., edge sites, pixcls, elements in a depth array) yield label likelihoods. Likelihoods from several sources are combined consistently in abstraction-hierarchical label structures using a new, computationally simple procedure. The resulting label likelihoods are combined with a priori spatial knowledge encoded in a Markov random field (MRF). From the prior probabilities and the evidence-dependent combined likelihoods, the a posteriori distribution of the labelings is derived using Bayes' theorem. A new inference method, Highest Confidence First (HCF) estimation, is used to infer a unique labeling from the a posteriori distribution that is consistent with both prior knowledge and evidence. HCF compares favorably to previous techniques, all equivalent to some form of energy minimization or optimization, in finding a good MRF labeling. HCF is computationally efficient and predictable and produces better answers (lower energies) while exhibiting graceful degradation under noise and least commitment under inaccurate models. The technique generalizes to higher-level vision problems and other domains, and is demonstrated on problems of intensity edge detection and surface depth reconstruction. We propose a mathematical framework, data structures, and algorithms for combining disparate sources of visual information (observational and prior) consistently into a raster representation of the scene. The representation may contain both symbolic (segmentation) and quantitative (reconstruction) information. The computation is formulated as a labeling problem: for each image location (site) find the appropriate attribute or label (such as depth, or whether it is an object boundary) describing the corresponding portion of the scene. The work involves the representation of knowledge, reasoning procedures for combining distinct bodies of knowledge, and methods for deriving scene properties from available knowledge. We have successfully applied the framework to two instances of the labeling problem-boundary detection from irradiance data and surface reconstruction from irradiance and range data. The paper deals with the following questions. (1) How to integrate multimodal visual data in a hierarchically structured hypothesis space. (2) How to incorporate a priori spatial knowledge with visual observations. (3) How to compute the optimal solution given a nonconvex a posteriori probability distribution of the possible solutions. (4) How simultaneously to reconstruct and segment the surfaces in a three-dimensional scene using sparse depth measurements and intensity observations. The answers we propose to the four questions are, respectively, the following: 1. A computationally simple, probabilistically justified procedure that provides consistent and coherent integration of early visual observations using hierarchically structured label trees.
Chou and Brown REF used highest confidence first (HCF) to infer a unique labeling from the posteriori distribution that is consistent with both the prior knowledge and evidence.
The theory and practice of Bayesian image labeling
International Journal of Computer Vision
International Journal of Computer Vision
Computer Science