OpenRaiser/Intern-Atlas huggingface dataset:
An Affine Invariant Tensor Dissimilarity Measure and Its Applications to Tensor-Valued Image Segmentation
Authors: Zhizhou Wang, Baba C. Vemuri
Venue: CVPR
Year: 2004

Tensor fields specifically, matrix valued data sets, have recently attracted increased attention in the fields of image processing, computer vision, visualization and medical imaging. In this paper, we present a novel definition of tensor "distance" grounded in concepts from information theory and incorporate it in the segmentation of tensor-valued images. In some applications, a symmetric positive definite (SPD) tensor at each point of a tensor valued image can be interpreted as the covariance matrix of a local Gaussian distribution. Thus, a natural measure of dissimilarity between SPD tensors would be the KL divergence or its relative. We propose the square root of the J-divergence (symmetrized KL) between two Gaussian distributions corresponding to the tensors being compared that leads to a novel closed form expression. Unlike the traditional Frobenius norm-based tensor distance, our "distance" is affine invariant, a desirable property in many applications. We then incorporate this new tensor "distance" in a region based active contour model for bimodal tensor field segmentation and show its application to the segmentation of diffusion tensor magnetic resonance images (DT-MRI) as well as for the texture segmentation problem in computer vision. Synthetic and real data experiments are shown to depict the performance of the proposed model.
From Fragments to Salient Closed Boundaries: An In-Depth Study
Authors: Song Wang, Jun Wang, Toshiro Kubota
Venue: CVPR
Year: 2004

This paper conducts an in-depth study on a classical perceptual-organization problem: finding salient closed boundaries from a set of boundary fragments detected in a noisy image. In this problem, a saliency boundary is formed by identifying and connecting a subset of fragments according to the simple Gestalt laws of closure, continuity, and proximity. Our specific interest is focused on the methods that aim to achieve boundary closure, an important global property of perceptual salient boundaries. In this paper, we analyze and compare three such methods that are developed in recent years: (a) Elder and Zucker's method based on the shortest-path algorithm, (b) Williams and Thornber's method combining the spectral-analysis and the strongly-connected-component algorithms, and (c) Wang, Kubota, and Siskind's method based on ratio-contour algorithm. Both theoretic analysis and experimental study show that, with a unified setting of fragment saliency, Wang, Kubota and Siskind's method more appropriately constrains the search space for the closed boundaries, and usually produces better performance than or at least comparable performance as the other two methods. Particularly, Wang, Kubota, and Siksind's method can always guarantee the boundary closure and simplicity, which may not be always hold in the other two methods. We construct and collect a variety of synthesized and real images for this comparison.
Region-Based Progressive Stereo Matching
Authors: Yichen Wei, Long Quan
Venue: CVPR
Year: 2004

A novel region-based progressive stereo matching algorithm is presented. It combines the strengths of previous region-based and progressive approaches. The progressive framework avoids the time consuming global optimization, while the inherent problem, the sensitivity to early wrong decisions, is significantly alleviated via the region-based representation. A growing-like process matches the regions progressively using a global best-first strategy based on a cost function integrating disparity smoothness and visibility constraint. The performance on standard evaluation platform using various real images shows that the algorithm is among the state-of-the-art both in accuracy and efficiency.
Space-Time Video Completion
Authors: Yonatan Wexler, Eli Shechtman, Michal Irani
Venue: CVPR
Year: 2004

We present a method for space-time completion of large space-time "holes" in video sequences of complex dynamic scenes. The missing portions are filled-in by sampling spatio-temporal patches from the available parts of the video, while enforcing global spatio-temporal consistency between all patches in and around the hole. This is obtained by posing the task of video completion and synthesis as a global optimization problem with a well-defined objective function. The consistent completion of static scene parts simultaneously with dynamic behaviors leads to realistic looking video sequences. Space-time video completion is useful for a variety of tasks, including, but not limited to: (i) Sophisticated video removal (of undesired static or dynamic objects) by completing the appropriate static or dynamic background information, (ii) Correction of missing/corrupted video frames in old movies, and (iii) Synthesis of new video frames to add a visual story, modify it, or generate a new one. Some examples of these are shown in the paper.
Feature Selection for Classifying High-Dimensional Numerical Data
Authors: Yimin Wu, Aidong Zhang 0001
Venue: CVPR
Year: 2004

Classifying high-dimensional numerical data is a very challenging problem. In high dimensional feature spaces, the performance of supervised learning methods suffers from the curse of dimensionality, which degrades both classification accuracy and efficiency. To address this issue, we present an efficient feature selection method to facilitate classifying high-dimensional numerical data. Our method employs balanced information gain to measure the contribution of each feature (for data classification); and it calculates feature correlation with a novel extension of balanced information gain. By integrating feature contribution and correlation, our feature selection approach uses a forward sequential selection algorithm to select uncorrelated features with large balanced information gain. Extensive experiments have been carried out on image and gene microarray datasets to demonstrate the effectiveness and robustness of the presented method.
Motion Layer Extraction in the Presence of Occlusion Using Graph Cut
Authors: Jiangjian Xiao, Mubarak Shah
Venue: CVPR
Year: 2004

Extracting layers from video is very important for video representation, analysis, compression, and recognition. Assuming that a scene can be approximately described by multiple planar regions, this paper describes a robust novel approach to automatically extract a set of affine transformations induced by these regions, and accurately segment the scene into several motion layers. First, a number of seed regions are determined by using two frame correspondences. Then the seed regions are expanded and refined using the level set representation and employing graph cut method. Next, these initial regions are merged into several initial layers according to the motion similarity. Third, after exploiting the occlusion order constraint on multiple frames the robust layer extraction is obtained by graph cut algorithm, and the occlusions between the overlapping layers are explicitly determined. Several examples are demonstrated in the experiments to show that our approach is effective and robust.
A Discriminative Learning Framework with Pairwise Constraints for Video Object Classification
Authors: Rong Yan, Jian Zhang 0003, Jie Yang 0001, Alexander G. Hauptmann
Venue: CVPR
Year: 2004

To deal with the problem of insufficient labeled data in video object classification, one solution is to utilize additional pairwise constraints that indicate the relationship between two examples, i.e., whether these examples belong to the same class or not. In this paper, we propose a discriminative learning approach which can incorporate pairwise constraints into a conventional margin-based learning framework. Different from previous work that usually attempts to learn better distance metrics or estimate the underlying data distribution, the proposed approach can directly model the decision boundary and, thus, require fewer model assumptions. Moreover, the proposed approach can handle both labeled data and pairwise constraints in a unified framework. In this work, we investigate two families of pairwise loss functions, namely, convex and nonconvex pairwise loss functions, and then derive three pairwise learning algorithms by plugging in the hinge loss and the logistic loss functions. The proposed learning algorithms were evaluated using a people identification task on two surveillance video data sets. The experiments demonstrated that the proposed pairwise learning algorithms considerably outperform the baseline classifiers using only labeled data and two other pairwise learning algorithms with the same amount of pairwise constraints.
Perceptual Organization of Radial Symmetries
Authors: Qing Yang 0002, Bahram Parvin
Venue: CVPR
Year: 2004

Radial symmetry is an important perceptual cue for the feature-based representation, fixation, and description of large-scale data sets. A new approach based on iterative voting along the gradient direction is introduced for inferring the center of mass for objects demonstrating radial symmetries that are not limited to convex geometries. The kernel topography is unique in that it votes for the most likely set of grid points where the center of mass may be located. Initially, it is applied in the direction of the gradient and then reoriented iteratively in the most probable direction. This technique can detect perceptual symmetries, has an excellent noise immunity, and is shown to be tolerant to moderate perturbation in scale. Applications of this approach to blobs with incomplete and noisy boundaries, multimedia scenes, and scientific images are demonstrated.
Covariance-Driven Mosaic Formation from Sparsely-Overlapping Image Sets with Application to Retinal Image Mosaicing
Authors: Gehua Yang, Charles V. Stewart
Venue: CVPR
Year: 2004

A new technique is presented for mosaicing sparsely-overlapping image sets, with a target application of assisting the diagnosis and treatment of retinal diseases. The geometric image transformations required to construct the mosaics are estimated by (1) estimating the transformations between as many pairs of images as possible, (2) extracting sets of constraints (correspondences)from the successfully registered image pairs, and (3) using these constraint sets to simultaneously (jointly) estimate the final transformations. Unfortunately, this may not be sufficient to construct seamless mosaics when two images overlap but can not be successfully registered (step 1). This paper presents a new method to generate constraints between such image pairs, and use these constraints to estimate a more consistent set of transformations. For each pair transformation parameter covariance matrices are computed and used to estimate the mapping error covariance matrices for individual features from one image. These features are matched in the second image by minimizing the resulting Mahalanobis distance. The generated correspondences are validated using robust estimation techniques and used to refine the estimates. The steps of covariance computation, matching, and transform estimation are repeated for all relevant image pairs until the final alignment converges. Results are presented and evaluated for several difficult image sets to illustrate the efficacy of the techniques.
Efficient Model-Based Linear Head Motion Recovery from Movies
Authors: Jian Yao 0001, Wai-kuen Cham
Venue: CVPR
Year: 2004

We propose an efficient method that estimates the motion parameters of a human head from a video sequence by using a three-layer linear iterative process. In the innermost layer, we estimate the motion of each input face image in a video sequence based on a generic face model and a small set of feature points. A fast iterative least-square method is used to recover these motion parameters. After that, we iteratively estimate three model scaling factors using multiple frames with the recovered poses in the middle layer. Finally, we update 3D coordinates of the feature points on the generic face model in the outer-most layer. Since all iterative processes can be solved linearly, the computational cost is low. Tests on synthetic data under noisy conditions and two real video sequences have been performed. Experimental results show that the proposed method is robust and has good performance.
From Facial Expression to Level of Interest: A Spatio-Temporal Approach
Authors: Mohammed Yeasin, Baptiste Bullot, Rajeev Sharma
Venue: CVPR
Year: 2004

This paper presents a novel approach to recognize the six universal facial expressions from visual data and use them to derive the level of interest using psychological evidences. The proposed approach relies on a two-step classification built on the top of refined optical flow computed from sequence of images. First, a bank of linear classifier was applied at frame level and the output of this stage was coalesced to produce a temporal signature for each observation. Second, temporal signatures thus computed from the training data set were used to train discrete hidden Markov models (HMMs) to learn the underlying models for each universal facial expressions. The average recognition rate of the proposed facial expression classifier is 90.9% without classifier fusion and 91.2% with fusion using a five fold cross validation scheme on a database of 488 video sequences that include 97 subjects. Recognized facial expressions were combined with the intensity of activity (motion) around the apex frame to measure the level of interest. To further illustrate the efficacy of the proposed approach two set of experiments, namely, television (TV) broadcast data (108 sequences of facial expression containing severe lighting conditions, diverse subjects and expressions) analysis and emotion elicitation on 21 subjects were conducted.
Searching the Web with Mobile Images for Location Recognition
Authors: Tom Yeh, Konrad Tollmar, Trevor Darrell
Venue: CVPR
Year: 2004

We describe an approach to recognizing location from mobile devices using image-based Web search. We demonstrate the usefulness of common image search metrics applied on images captured with a camera-equipped mobile device to find matching images on the World Wide Web or other general-purpose databases. Searching the entire Web can be computationally overwhelming, so we devise a hybrid image-and-keyword searching technique. First, image-search is performed over images and links to their source Web pages in a database that indexes only a small fraction of the Web. Then, relevant keywords on these Web pages are automatically identified and submitted to an existing text-based search engine (e.g. Google) that indexes a much larger portion of the Web. Finally, the resulting image set is filtered to retain images close to the original query. It is thus possible to efficiently search hundreds of millions of images that are not only textually related but also visually relevant. We demonstrate our approach on an application allowing users to browse Web pages matching the image of a nearby location.
Asymmetrically Boosted HMM for Speech Reading
Authors: Pei Yin, Irfan A. Essa, James M. Rehg
Venue: CVPR
Year: 2004

Speech reading, also known as lip reading, is aimed at extracting visual cues of lip and facial movements to aid in recognition of speech. The main hurdle for speech reading is that visual measurements of lip and facial motion lack information-rich features like the Mel frequency cepstral coefficients (MFCC), widely used in acoustic speech recognition. These MFCC are used with hidden Markov models (HMM) in most speech recognition systems at present. Speech reading could greatly benefit from automatic selection and formation of informative features from measurements in the visual domain. These new features can then be used with HMM to capture the dynamics of lip movement and eventual recognition of lip shapes. Towards this end, we use AdaBoost methods for automatic visual feature formation. Specifically, we design an asymmetric variant of AdaBoost M2 algorithm to deal with the ill-posed multi-class sample distribution inherent in our problem. Our experiments show that the boosted HMM approach outperforms conventional AdaBoost and HMM classifiers. Our primary contributions are in the design of (a) boosted HMM and (b) asymmetric multi-class boosting.
Using Skew Gabor Filter in Source Signal Separation and Local Spectral Multi-Orientation Analysis
Authors: Weichuan Yu, Gerald Sommer, Kostas Daniilidis
Venue: CVPR
Year: 2004

Responses of Gabor wavelets in the mid-frequency space build a local spectral representation scheme with optimal properties regarding the time-frequency uncertainty principle. However, when using Gabor wavelets we observe a skewness in the mid-frequency space caused by the unsymmetrically spreading effect of Gabor wavelets. Though in most current applications the skewness does not obstruct the sampling of the spectral domain, it affects the identification and separation of source signals from the filter response in the mid-frequency space. In this paper, we present a modification of the original Gabor filter, the skew Gabor filter, to correct the skewness so that the filter responses can be described with a sum-of-Gaussians model. The correction enables us to use higher-order-moment information to analytically separate different source signal components. This provides us with an analytical framework to overcome the limited spectral resolution of other local spectral representations. Examples in source signal separation and local spectral multi-orientation analysis are shown.
Recovering Shape and Reflectance Model of Non-Lambertian Objects from Multiple Views
Authors: Tian-Li Yu 0002, Ning Xu 0005, Narendra Ahuja
Venue: CVPR
Year: 2004

This paper proposes an algorithm to simultaneously estimate both the 3D shape and parameters of a surface reflectance model from multiple views of an object made of a single material. The algorithm is based on a multiple view shape from shading method. A triangular mesh represents the shape of the object. The Phong reflectance model is used to model the surface reflectance. We iteratively find the shape and reflectance parameters that best fit all input images. Subdividing triangles in the mesh into smaller ones gradually refines the estimates of shape and reflectance model. The estimation takes into account both self-occlusion and self-shadowing. Analysis shows that the accuracy of reflectance estimation is limited by the triangle size in the shape model. We also propose to use Richardson extrapolation to overcome this and further refine the reflectance model estimate. The estimated 3D shape and reflectance model can be used to render the same object from different viewing directions and under different lighting conditions. Experimental results on both synthetic and real objects are given.
Pointwise Motion Tracking in Echocardiographic Images
Authors: Weichuan Yu, Ping Yan, Albert J. Sinusas, Karl Thiele, James S. Duncan
Venue: CVPR
Year: 2004

We study the problem of pointwise motion tracking in echocardiographic images. We show that decorrelation between tissue motion and intensity variation is inevitable for certain kinds of tissue motion and decorrelation compensation is an ill-posed inverse problem if the decorrelation is beyond a certain correlation threshold. We compare the performance of different features using simulations and phantom examples. We find a threshold value of correlation coefficients below which the B-Mode signal works better than the radio frequency (RF) signal in the analysis of large deformation. We also demonstrate that the introduction of a quantitative reliability measure helps to improve the robustness of displacement estimation.
Affine Image Registration Using a New Information Metric
Authors: Jie Zhang 0025, Anand Rangarajan 0001
Venue: CVPR
Year: 2004

We present a new information metric for multimodality image registration. The metric is technically a pseudometric since it satisfies the properties, (i) nonnegativity, (ii) symmetry, (iii) triangle inequality and is (iv) zero if (but not only if) the two image intensities are identical. Information metrics are rarely used in image registration and notably, the widely used mutual information measure is not a metric. Given images A and B, the metric used here is the sum of the conditional entropies H(A/spl bsol/B) and H(B/spl bsol/A). We show that when compared to mutual information which can even become negative in the multiple image case, it is easier to extend our metric to the registration of multiple images. And, after using an upper bound, we show that the sum of the conditional entropies can be efficiently computed even in the multiple image case. We use the metric to simultaneously register multiple 2D slice images obtained from proton density (PD), magnetic resonance (MR) T2 and MR T1 3D volumes and to match human face images obtained wider different illuminations. Our results demonstrate the efficacy of the metric in affine, multiple image registration.
Restoration of Curved Document Images through 3D Shape Modeling
Authors: Zheng Zhang 0003, Chew Lim Tan, Liying Fan
Venue: CVPR
Year: 2004

In this paper, we address the problem of discovering the 3D shape of a book surface from the shading information in a scanned document image. This shape-from-shading problem is characterized in real world environments by a proximal and a moving light source, Lambertian reflection and a non-uniform albedo distribution. By considering all these factors, we first build the practical model (consists of geometric model and optical model) to reconstruct the 3D shape of book surface. We next restore the scanned image using this shape based on two models, namely de-shading and dewarping models. Finally, we compare the OCR results on the original and restored document image. The experiments show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.
Registration of Diffusion Tensor Images
Authors: Hui Zhang 0005, Paul A. Yushkevich, James C. Gee
Venue: CVPR
Year: 2004

This paper presents a novel affine registration algorithm for diffusion tensor images. The proposed metric derived from the standpoint of diffusion profiles not only has concrete physical underpinning but also can be extended for comparing higher-order diffusion models. The non-translational part of the affine transformation is parametrized in the spirit of the Polar Decomposition Theorem. The registration objective function and its derivatives are derived analytically by combining this parametrization scheme with finite strain tensor reorientation. The affine algorithm is embeded in a multi-resolution piecewise affine framework for non-rigid registration.
Hidden Semantic Concept Discovery in Region Based Image Retrieval
Authors: Ruofei Zhang, Zhongfei (Mark) Zhang
Venue: CVPR
Year: 2004

This work addresses content based image retrieval (CBIR), focusing on developing a hidden semantic concept discovery methodology to address effective semantics-intensive image retrieval. In our approach, each image in the database is segmented to region; associated with homogenous color, texture, and shape features. By exploiting regional statistical information in each image and employing a vector quantization method, a uniform and sparse region-based representation is achieved. With this representation a probabilistic model based on statistical-hidden-class assumptions of the image database is obtained, to which expectation-maximization (EM) technique is applied to analyze semantic concepts hidden in the database. An elaborated retrieval algorithm is designed to support the probabilistic model. The semantic similarity is measured through integrating the posterior probabilities of the transformed query image, as well as a constructed negative example, to the discovered semantic concepts. The proposed approach has a solid statistical foundation and the experimental evaluations on a database of 10,000 general-purposed images demonstrate its promise of the effectiveness.