Two-way Poisson mixture models for simultaneous document classification and word clustering
An approach to simultaneous document classification and word clustering is developed using a two-way mixture model of Poisson distributions. Each document is represented by a vector with each dimension specifying the number of occurrences of a particular word in the document in question. As a collection of documents across several classes usually makes use of a large number of words, the document vectors are of high dimension. On the other hand, the number of distinct words in any single document is usually substantially smaller than the size of the vocabulary, leading to sparse document vectors. A mixture of Poisson distributions is used to model the multivariate distribution of the word counts in the documents within each class. To address the issues of high dimensionality and sparsity, the parameters in the mixture model are regularized by imposing a clustering structure on the set of words. An EM-style algorithm for the two-way mixture model will be derived for parameter estimation with the clustering of words part of the estimation process. The connection of the two-way mixture model with dimension reduction will also be elucidated. Experiments on the newsgroup data have demonstrated promising results.

title: A source coding approach to classification by vector quantization and the principle of minimum description length
abstract: An algorithm for supervised classification using vector quantization and entropy coding is presented. The classification rule is formed from a set of training data {(X/sub i/, Y/sub i/)}/sub i=1//sup n/, which are independent samples from a joint distribution P/sub XY/. Based on the principle of minimum description length (MDL), a statistical model that approximates the distribution P/sub XY/ ought to enable efficient coding of X and Y. On the other hand, we expect a system that encodes (X, Y) efficiently to provide ample information on the distribution P/sub XY/. This information can then be used to classify X, i.e., to predict the corresponding Y based on X. To encode both X and Y, a two-stage vector quantizer is applied to X and a Huffman code is formed for Y conditioned on each quantized value of X. The optimization of the encoder is equivalent to the design of a vector quantizer with an objective function reflecting the joint penalty of quantization error and misclassification rate. This vector quantizer provides an estimation of the conditional distribution of Y given X, which in turn yields an approximation to the Bayes classification rule. This algorithm, namely discriminant vector quantization (DVQ), is compared with learning vector quantization (LVQ) and CART/sup R/ on a number of data sets. DVQ outperforms the other two on several data sets. The relation between DVQ, density estimation, and regression is also discussed.

title: Real-Time Computerized Annotation of Pictures
abstract: Developing effective methods for automated annotation of digital pictures continues to challenge computer scientists. The capability of annotating pictures by computers can lead to breakthroughs in a wide range of applications, including Web image search, online picture-sharing communities, and scientific experiments. In this work, the authors developed new optimization and estimation techniques to address two fundamental problems in machine learning. These new techniques serve as the basis for the automatic linguistic indexing of pictures - real time (ALIPR) system of fully automatic and high-speed annotation for online pictures. In particular, the D2-clustering method, in the same spirit as K-Means for vectors, is developed to group objects represented by bags of weighted vectors. Moreover, a generalized mixture modeling technique (kernel smoothing as a special case) for nonvector data is developed using the novel concept of hypothetical local mapping (HLM). ALIPR has been tested by thousands of pictures from an Internet photo-sharing site, unrelated to the source of those pictures used in the training process. Its performance has also been studied at an online demonstration site, where arbitrary users provide pictures of their choices and indicate the correctness of each annotation word. The experimental results show that a single computer processor can suggest annotation terms in real time and with good accuracy.

title: Unsupervised Multiresolution Segmentation for Images with Low Depth of Field
abstract: Unsupervised segmentation of images with low depth of field (DOF) is highly useful in various applications. This paper describes a novel multiresolution image segmentation algorithm for low DOF images. The algorithm is designed to separate a sharply focused object-of-interest from other foreground or background objects. The algorithm is fully automatic in that all parameters are image independent. A multi-scale approach based on high frequency wavelet coefficients and their statistics is used to perform context-dependent classification of individual blocks of the image. Unlike other edge-based approaches, our algorithm does not rely on the process of connecting object boundaries. The algorithm has achieved high accuracy when tested on more than 100 low DOF images, many with inhomogeneous foreground or background distractions. Compared with he state of the art algorithms, this new algorithm provides better accuracy at higher speed.

title: Asymptotic Performance of Vector Quantizers with a Perceptual Distortion Measure
abstract: Gersho's (1979) bounds on the asymptotic performance of vector quantizers are valid for vector distortions which are powers of the Euclidean norm. Yamada, Tazaki, and Gray (1980) generalized the results to distortion measures that are increasing functions of the norm of their argument. In both cases, the distortion is uniquely determined by the vector quantization error, i.e., the Euclidean difference between the original vector and the codeword into which it is quantized. We generalize these asymptotic bounds to input-weighted quadratic distortion measures and measures that are approximately output-weighted-quadratic when the distortion is small, a class of distortion measures often claimed to be perceptually meaningful. An approximation of the asymptotic distortion based on Gersho's conjecture is derived as well. We also consider the problem of source mismatch, where the quantizer is designed using a probability density different from the true source density. The resulting asymptotic performance in terms of distortion increase in decibels is shown to be linear in the relative entropy between the true and estimated probability densities.

title: On Zador's entropy-constrained quantization theorem
abstract: Zador's classic result for the asymptotic high-rate behavior of entropy-constrained vector quantization is recast in a Lagrangian form which better matches the Lloyd algorithm used to optimize such quantizers. A proof that the result holds for a general class of distributions is sketched.

title: Computing Distance Maps Efficiently Using an Optical Bus
abstract: This paper discusses an algorithm for finding a distance map for an image efficiently using an optical bus. The computational model considered is the arrays with a reconfigurable pipelined bus system (LARPBS), which is introduced recently based on current electronic and optical technologies. It is shown that the problem for an n × n image can be implemented in O(log n log log n) time deterministically or in O(log n) time with high probability on an LARPBS with n2 processors. We also show that the problem can be solved in O(log log n) time deterministically or in O(1) time with high probability on an LARPBS with n3 processors. The algorithm compares favorably to the best known parallel algorithms for the same problem in the literature.

title: Modeling the effects of elevation data resolution on the performance of topography-based watershed runoff simulation
abstract: The spatial uncertainty of a topography based rainfall runoff model (TOPMODEL) is addressed in this study to assess its variability in simulating watershed hydrologic response with regards to the change of digital elevation model (DEM) resolution. Twelve DEM realizations of different grid sizes ranging from 30m to 3000m for each of two case watersheds are used for comparative examinations. The study shows that DEM grid size has significant influence on the topographic index distribution which represents the effect of topography on watershed hydrology in TOPMODEL. The smoothing effect of grid size increase may result in deteriorated topographic index distributions at coarse resolutions as the ratio of grid cell area to watershed area gets larger. The simulated discharges and model efficiencies using a same set of TOPMODEL parameters are sensitive to DEM grid size especially at coarse resolutions. This sensitivity, however, can be moderated by parameter calibrations as the optimization runs show that fairly equal efficiencies can be preserved by the compensation effect of transmissivity parameter T"0 within a large extent of DEM resolution for each watershed. The interaction between T"0 and the topographic index distribution with respect to TOPMDOEL model performance is also examined. It is found that both study watersheds demonstrate a similar pattern of change in model performance along with the increase of the grid-to-watershed ratio. The analysis reveals that the ratio poses an important factor in controlling the effect of DEM grid size on TOPMODEL performance. A ratio of less than 5% is suggested in DEM resolution selection for TOPMODEL applications based on the results of this study.

title: Quality Evaluation in Flexible Machining Systems: A Flexible Fixture Case Study
abstract: Most of the studies in flexible manufacturing systems address the issues of flexibility, productivity, cost, etc. The impact of flexible lines on product quality is less studied. This paper presents a quantitative model based on Markov chain analysis to evaluate quality performance of a flexible machining system. A case study of flexible fixture is provided to illustrate the applicability of the method.

title: Rule Evaluations, Attributes, and Rough Sets: Extension and a Case Study
abstract: Manually evaluating important and interesting rules generated from data is generally infeasible due to the large number of rules extracted. Different approaches such as rule interestingness measures and rule quality measures have been proposed and explored previously to extract interesting and high quality association rules and classification rules. Rough sets theory was originally presented as an approach to approximate concepts under uncertainty. In this paper, we explore rough sets based rule evaluation approaches in knowledge discovery. We demonstrate rule evaluation approaches through a real-world geriatric care data set from Dalhousie Medical School. Rough set based rule evaluation approaches can be used in a straightforward way to rank the importance of the rules. One interesting system developed along these lies in HYRIS (HYbrid Rough sets Intelligent System). We introduce HYRIS through a case study on survival analysis using the geriatric care data set.

title: Multi-channel Noise Reduction in Noisy Environments
abstract: Multi-channel noise reduction has been widely researched to reduce acoustic noise signals and to improve the performance of many speech applications in noisy environments. In this paper, we first introduce the state-of-the-art multi-channel noise reduction methods, especially beamforming based methods, and discuss their performance limitations. Subsequently, we present a multi-channel noise reduction system we are developing that consists of localized noise suppression by microphone array and non-localized noise suppression by post-filtering. Experimental results are also presented to show the benefits of our developed noise reduction system with respect to the traditional algorithms in terms of speech recognition rate. Some suggestions are finally presented for the further research.