improving feature space based image segmentation via density modification feature space based approaches have been popularly used to perform low level image analysis in this paper a density modification framework that enhances density map based discriminability of feature values in a feature space is proposed in order to aid feature space based segmentation in images the framework embeds a position dependent property associated with each sample in the feature space of an image into the corresponding density map and hence modifies it the property association and embedding operations in the framework is implemented using a fuzzy set theory based system devised with cues from beam theory of solid mechanics and the appropriateness of this approach is established qualitative and quantitative experimental results of segmentation in images are given to demonstrate the effectiveness of the proposed density modification framework and the usefulness of feature space based segmentation via density modification

simplifying mixture models through function approximation the finite mixture model is widely used in various statistical learning problems however the model obtained may contain a large number of components making it inefficient in practical applications in this paper we propose to simplify the mixture model by minimizing an upper bound of the approximation error between the original and the simplified model under the use of the l 2 distance measure this is achieved by first grouping similar components together and then performing local fitting through function approximation the simplified model obtained can then be used as a replacement of the original model to speed up various algorithms involving mixture models during training e g bayesian filtering belief propagation and testing e g kernel density estimation support vector machine svm testing encouraging results are observed in the experiments on density estimation clustering based image segmentation and simplification of svm decision functions

fast parzen density estimation using clustering based branch and bound this correspondence proposes a fast parzen density estimation algorithm that would be especially useful in nonparametric discriminant analysis problems by preclustering the data and applying a simple branch and bound procedure to the clusters significant numbers of data samples that would contribute little to the density estimate can be excluded without detriment to actual evaluation via the kernel functions this technique is especially helpful in the multivariant case and does not require a uniform sampling grid the proposed algorithm may also be used in conjunction with the data reduction technique of fukunaga and hayes 1989 to further reduce the computational load experimental results are presented to verify the effectiveness of this algorithm

block transmission over dispersive channels transmit filter optimization and realization and mmse dfe receiver performance optimal transmit filters for packet based data transmission on dispersive gaussian noise linear time invariant channels are derived by maximizing the mutual information subject to a fixed input power budget a quasi stationary approximation to the optimal nonstationary input covariance process is derived and shown to exhibit negligible mutual information loss from the optimal case for situations of most practical interest moreover this quasi stationary approximation results in efficiently computed lattice or pole zero implementations of the transmit filter by considering the popular finite impulse response minimum mean square error decision feedback equalizer fir mmse dfe as a receiver structure we show that transmitter optimization results in an appreciable improvement in the decision point signal to noise ratio finally we show that as the output blocklength becomes infinite the optimum finite dimensional nonstationary input covariance process converges to a stationary process whose power spectrum obeys the well known water pouring distribution

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

exact mean integrated squared error an exact and easily computable expression for the mean integrated squared error mise for the kernel estimator of a general normal mixture density is given for gaussian kernels of arbitrary order this provides a powerful new way of understanding density estimation which complements the usual tools of simulation and asymptotic analysis the family of normal mixture densities is very flexible and the formulae derived allow simple exact analysis for a wide variety of density shapes a number of applications of this method giving important new insights into kernel density estimation are presented among these is the discovery that the usual asymptotic approximations to the mise can be quite inaccurate especially when the underlying density contains substantial fine structure and also strong evidence that the practical importance of higher order kernels is surprisingly small for moderate sample sizes 1 introduction substantial research has been devoted to kernel density estimation this is because it provides a simple yet appealing context in which to study problems and issues that arise in all types of nonparametric curve estimation this includes regression spectral density and hazard estimation and also a variety of other estimators including histograms splines and orthogonal series three important and useful tools for understanding the behavior of nonparametric curve estimators are asymptotic analysis simulation and numerical calculation of error criteria each of these methods provides many useful insights into the complicated structure present in the study of curve estimation however each has its limitations as well rhe strength of asymptotic analysis is that it frequently allows simultaneous study of many different specific examples through general results applying to entire classes of settings the weakness of asymptotics is that they only describe behavior in the limit this is still very useful in many situations because the asymptotics describe the actual situation quite well however it is less useful when the asymptotics have not yet kicked in that is in studying situations where the asymptotically dominant effect has not taken over yet perhaps the biggest drawback to asymptotics is that it is very difficult to determine in a given situation which of these possibilities is occurring

recent developments in nonparametric density estimation advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research and especially the development of nonparametric data analysis procedures in particular theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics such as nonparametric regression nonparametric discrimination and nonparametric pattern recognition this article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject the early density estimation methods such as the histogram kernel estimators and orthogonal series estimators are still very popular and recent research on them is described different types of restricted maximum likelihood density estimators including order restricted estimators maximum penalized likelihood estimators and sieve estimators are discussed where restrictions are imposed upon the class of densities or on the form of the likelihood function nonparametric density estimators that are data adaptive and lead to locally smoothed estimators are also discussed these include variable partition histograms estimators based on statistically equivalent blocks nearest neighbor estimators variable kernel estimators and adaptive kernel estimators for the multivariate case extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive a method of multivariate density estimation that did not spring from a univariate generalization is described namely projection pursuit density estimation in which both dimensionality reduction and density estimation can be pursued at the same time finally some areas of related research are mentioned such as nonparametric estimation of functionals of a density robust parametric estimation semiparametric models and density estimation for censored and incomplete data directional and spherical data and density estimation for dependent sequences of observations

linear mmse multiuser receivers mai conditional weak convergence and network capacity we explore the performance of minimum mean square error mmse multiuser receivers in wireless systems where the signatures are modeled as random and take values in complex space first we study the conditional distribution of the output multiple access interference mai of the mmse receiver by appealing to the notion of conditional weak convergence we find that the conditional distribution of the output mai given the received signatures and received powers converges in probability to a proper complex gaussian distribution that does not depend on the signatures this result indicates that in a large system the output interference of the mmse receiver is approximately gaussian with high probability and that systems with mmse receivers are robust to the randomness of the signatures building on the gaussianity of the output interference we then take the quality of service qos requirements as meeting the signal to interference ratio sir constraints and identify the network capacity of single class systems with random spreading the network capacity is expressed uniquely in terms of the sir requirements and received power distributions compared to the network capacity corresponding to the optimal signature allocation we conclude that at the cost of transmission power the gap between the network capacity corresponding to optimal signatures and that corresponding to random signatures can be made arbitrarily small therefore from the viewpoint of the network capacity systems with mmse receivers are robust to the randomness of signatures

an algorithm for data driven bandwidth selection the analysis of a feature space that exhibits multiscale patterns often requires kernel estimation techniques with locally adaptive bandwidths such as the variable bandwidth mean shift proper selection of the kernel bandwidth is however a critical step for superior space analysis and partitioning this paper presents a mean shift based approach for local bandwidth selection in the multimodal multivariate case the method is based on a fundamental property of normal distributions regarding the bias of the normalized density gradient this paper demonstrates that within the large sample approximation the local covariance is estimated by the matrix that maximizes the magnitude of the normalized mean shift vector using this property the paper develops a reliable algorithm which takes into account the stability of local bandwidth estimates across scales the validity of the theoretical results is proven in various space partitioning experiments involving the variable bandwidth mean shift

what is a model of the lambda calculus an elementary purely algebraic definition of model for the untyped lambda calculus is given this definition is shown to be equivalent to the natural semantic definition based on environments these definitions of model are consistent with and yield a completeness theorem for the standard axioms for lambda convertibility a simple construction of models for lambda calculus is reviewed the algebraic formulation clarifies the relation between combinators and lambda terms

an information fusion framework for robust shape tracking existing methods for incorporating subspace model constraints in shape tracking use only partial information from the measurements and model distribution we propose a unified framework for robust shape tracking optimally fusing heteroscedastic uncertainties or noise from measurement system dynamics and a subspace model the resulting nonorthogonal subspace projection and fusion are natural extensions of the traditional model constraint using orthogonal projection we present two motion measurement algorithms and introduce alternative solutions for measurement uncertainty estimation we build shape models offline from training data and exploit information from the ground truth initialization online through a strong model adaptation our framework is applied for tracking in echocardiograms where the motion estimation errors are heteroscedastic in nature each heart has a distinct shape and the relative motions of epicardial and endocardial borders reveal crucial diagnostic features the proposed method significantly outperforms the existing shape space constrained tracking algorithm due to the complete treatment of heteroscedastic uncertainties the strong model adaptation and the coupled tracking of double contours robust performance is observed even on the most challenging cases

ber minimized ofdm systems with channel independent precoders we consider the minimization of uncoded bit error rate ber for the orthogonal frequency division multiplexing ofdm system with an orthogonal precoder we analyze the ber performance of precoded ofdm systems with zero forcing and minimum mean squared error mmse receivers in the case of mmse receivers we show that for quadrature phase shift keying qpsk there exists a class of optimal precoders that are channel independent examples of this class include the discrete fourier transform dft matrix and the hadamard matrix when the precoder is the dft matrix the resulting optimal transceiver becomes the single carrier system with cyclic prefix sc cp system we also show that the worst solution corresponds to the conventional ofdm system the conventional ofdm system has the largest ber in the case of zero forcing receivers the design of optimal transceiver depends on the signal to noise ratio snr for higher snr solutions of optimal precoders are the same as those of mmse receivers

kernel based object tracking a new approach toward target representation and localization the central component in visual tracking of nonrigid objects is proposed the feature histogram based target representations are regularized by spatial masking with an isotropic kernel the masking induces spatially smooth similarity functions suitable for gradient based optimization hence the target localization problem can be formulated using the basin of attraction of the local maxima we employ a metric derived from the bhattacharyya coefficient as similarity measure and use the mean shift procedure to perform the optimization in the presented tracking examples the new method successfully coped with camera motion partial occlusions clutter and target scale variations integration with motion filters and data association techniques is also discussed we describe only a few of the potential applications exploitation of background information kalman tracking using motion models and face tracking

cluster based probability model and its application to image and texture processing we develop analyze and apply a specific form of mixture modeling for density estimation within the context of image and texture processing the technique captures much of the higher order nonlinear statistical relationships present among vector elements by combining aspects of kernel estimation and cluster analysis experimental results are presented in the following applications image restoration image and texture compression and texture classification

real time vector quantization and clustering based on ordinary differential equations this brief presents a dynamical system approach to vector quantization or clustering based on ordinary differential equations with the potential for real time implementation two examples of different pattern clusters demonstrate that the model can successfully quantize different types of input patterns furthermore we analyze and study the stability of our dynamical system by discovering the equilibrium points for certain input patterns and analyzing their stability we have shown the quantizing behavior of the system with respect to its vigilance parameter the proposed system is applied to two real world problems providing comparable results to the best reported findings this validates the effectiveness of our proposed approach

gaussian mixture models reduction by variational maximum mutual information gaussian mixture models gmms are widely used in a variety of classification tasks where it is often important to approximate high order models by models with fewer components the paper proposes a novel approach to this problem based on a parametric realization of the maximum mutual information mmi criterion and its approximation by a closed form expression named variational mmi vmmi the maximization of the vmmi can be carried out in an analytically tractable manner and it aims at improving the discrimination ability of the reduced set of models a goal that was not targeted in previous approaches that simplify each class related gmm independently two effective algorithms are proposed and studied for the optimization of the vmmi criterion one is a steepest descent type algorithm and the other called line search mmb a functions lsaf uses concave associated functions experiments held in two speech related tasks phone recognition and language recognition demonstrate that the vmmi based parametric model reduction algorithms significantly outperform previous non discriminative methods according to these experiments the em like lsaf based algorithm requires less iterations and converges to a better value of the objective function compared to the steepest descent algorithm

output mai distributions of linear mmse multiuser receivers in ds cdma systems multiple access interference mai in a code division multiple access cdma system plays an important role in performance analysis and characterization of fundamental system limits we study the behavior of the output mai of the minimum mean square error mmse receiver employed in the uplink of a direct sequence ds cdma system we focus on imperfect power controlled systems with random spreading and establish that in a synchronous system 1 the output mai of the mmse receiver is asymptotically gaussian and 2 for almost every realization of the signatures and received powers the conditional distribution of the output mai converges weakly to the same gaussian distribution as in the unconditional case we also extend our study to asynchronous systems and establish the gaussian nature of the output interference these results indicate that in a large system the output interference is approximately gaussian and the performance of the mmse receiver is robust to the randomness of the signatures and received powers the gaussianity justifies the use of single user gaussian codes for cdma systems with linear mmse receivers and implies that from the viewpoints of detection and channel capacity signal to interference ratio sir is the key parameter that governs the performance of the mmse receiver in a cdma system

density weighted nystrom method for computing large kernel eigensystems the nystrom method is a well known sampling based technique for approximating the eigensystem of large kernel matrices however the chosen samples in the nystrom method are all assumed to be of equal importance which deviates from the integral equation that defines the kernel eigenfunctions motivated by this observation we extend the nystrom method to a more general density weighted version we show that by introducing the probability density function as a natural weighting scheme the approximation of the eigensystem can be greatly improved an efficient algorithm is proposed to enforce such weighting in practice which has the same complexity as the original nystrom method and hence is notably cheaper than several other alternatives experiments on kernel principal component analysis spectral clustering and image segmentation demonstrate the encouraging performance of our algorithm

precoded and vector ofdm robust to channel spectral nulls and with reduced cyclic prefix length in single transmit antenna systems the performance of orthogonal frequency division multiplexing ofdm systems may be degraded when intersymbol interference isi channels have spectral nulls also when isi channels have many taps the data rate overhead due to the insertion of the cyclic prefix is high we first propose a precoded ofdm system that may improve the performance of the ofdm systems for spectral null channels we also propose size k spl times x1 vector ofdm vofdm systems that reduce the cyclic prefix length by k times compared to the conventional ofdm systems the precoding scheme is simply to insert one or more zeros between each two sets of k consecutive information symbols although it can be generalized to a general form this precoding scheme may be able to remove the spectral nulls of an isi channel without knowing the isi channel when no zero is inserted between each two sets of k consecutive information symbols and only each k consecutive symbols are blocked together we obtain vofdm systems both theoretical performance analysis and simulation results are presented finally we compare the combination of the vofdm systems and the unitary matrix modulation with the conventional ofdm systems and the phase shift keying modulation where both differential noncoherent and coherent modulations and demodulations are considered

simplifying mixture models using the unscented transform mixture of gaussians mog model is a useful tool in statistical learning in many learning processes that are based on mixture models computational requirements are very demanding due to the large number of components involved in the model we propose a novel algorithm for learning a simplified representation of a gaussian mixture that is based on the unscented transform which was introduced for filtering nonlinear dynamical systems the superiority of the proposed method is validated on both simulation experiments and categorization of a real image database the proposed categorization methodology is based on modeling each image using a gaussian mixture model a category model is obtained by learning a simplified mixture model from all the images in the category

probability density estimation from optimally condensed data samples the requirement to reduce the computational cost of evaluating a point probability density estimate when employing a parzen window estimator is a well known problem this paper presents the reduced set density estimator that provides a kernel based density estimator which employs a small percentage of the available data sample and is optimal in the l sub 2 sense while only requiring spl oscr n sup 2 optimization routines to estimate the required kernel weighting coefficients the proposed method provides similar levels of performance accuracy and sparseness of representation as support vector machine density estimation which requires spl oscr n sup 3 optimization routines and which has previously been shown to consistently outperform gaussian mixture models it is also demonstrated that the proposed density estimator consistently provides superior density estimates for similar levels of data reduction to that provided by the recently proposed density based multiscale data condensation algorithm and in addition has comparable computational scaling the additional advantage of the proposed method is that no extra free parameters are introduced such as regularization bin width or condensation ratios making this method a very simple and straightforward approach to providing a reduced set density estimator with comparable accuracy to that of the full sample parzen density estimator

a new method for the nonlinear transformation of means and covariances in filters and estimators this paper describes a new approach for generalizing the kalman filter to nonlinear systems a set of samples are used to parametrize the mean and covariance of a not necessarily gaussian probability distribution the method yields a filter that is more accurate than an extended kalman filter ekf and easier to implement than an ekf or a gauss second order filter its effectiveness is demonstrated using an example

weighted parzen windows for pattern classification this paper introduces the weighted parzen window classifier the proposed technique uses a clustering procedure to find a set of reference vectors and weights which are used to approximate the parzen window kernel estimator classifier the weighted parzen window classifier requires less computation and storage than the full parzen window classifier experimental results showed that significant savings could be achieved with only minimal if any error rate degradation for synthetic and real data sets

retraction map categories and their applications to the construction of lambda calculus models this paper deals with categorical models of the calculus we generalize the inverse limit method scott used for his construction of d and introduce order enriched ccc s retraction map categories and categories an order enriched ccc is a cartesian closed category c equipped with a partial order relation on the set of the arrows a retraction map category of c is r r i j where is a partial order relation on the set c of all the objects of c r is the category of the poset c and i and j are functors from r to c and from r op to c that satisfy the conditions 1 j a b i a b id a and 2 i a b j a b id b for every arrow a b a b in r i e a b the category e e c r of c w r t r is the category whose objects are ideals of c and whose arrows are ideals of c where is the partial order relation in r and is the partial order relation defined by f g iff dom f dom g cod f cod g in r and f j a b g i a b in c we show that every category e e c r is also an order enriched ccc moreover when e and r satisfy a particular condition e c r has a reflexive object for example if there is an ideal u of c satisfying the following conditions then u is isomorphic to u u in e and a algebra is constructed from e and u 1 for every pair of a b u u contains b a and 2 for every c u there are a b u such that c b a we reconstruct p and d using categories

sparse density estimation on the multinomial manifold a new sparse kernel density estimator is introduced based on the minimum integrated square error criterion for the finite mixture model since the constraint on the mixing coefficients of the finite mixture model is on the multinomial manifold we use the well known riemannian trust region rtr algorithm for solving this problem the first and second order riemannian geometry of the multinomial manifold are derived and utilized in the rtr algorithm numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with an accuracy competitive with those of existing kernel density estimators

mapping from structure to dynamics a unified view of dynamical processes on networks although it is unambiguously agreed that structure plays a fundamental role in shaping the collective dynamics of complex systems how structure determines dynamics exactly still remains unclear we investigate a general computational transformation by which we can map the network topology directly to the dynamical patterns emergent on it independent of the nature of the dynamical processes remarkably we find that many seemingly different dynamical processes on networks such as coupled oscillators ensemble neuron firing epidemic spreading and diffusion can all be understood and unified through this same procedure utilizing the inherent multiscale nature of this structure dynamics transformation we further define a multiscale complexity measure which can quantify the functional diversity a general network can support at different organization levels using only its structure we find that a wide variety of topological features observed in real networks such as modularity hierarchy degree heterogeneity and mixing all result in higher complexity this result suggests that the demand for functional diversity is driving the structural evolution of physical networks

count data modeling and classification using finite mixtures of distributions in this paper we consider the problem of constructing accurate and flexible statistical representations for count data which we often confront in many areas such as data mining computer vision and information retrieval in particular we analyze and compare several generative approaches widely used for count data clustering namely multinomial multinomial dirichlet and multinomial generalized dirichlet mixture models moreover we propose a clustering approach via a mixture model based on a composition of the liouville family of distributions from which we select the beta liouville distribution and the multinomial the novel proposed model which we call multinomial beta liouville mixture is optimized by deterministic annealing expectation maximization and minimum description length and strives to achieve a high accuracy of count data clustering and model selection an important feature of the multinomial beta liouville mixture is that it has fewer parameters than the recently proposed multinomial generalized dirichlet mixture the performance evaluation is conducted through a set of extensive empirical experiments which concern text and image texture modeling and classification and shape modeling and highlights the merits of the proposed models and approaches

density estimation with confidence sets exemplified by superclusters and voids in the galaxies abstract a method is presented for forming both a point estimate and a confidence set of semiparametric densities the final product is a three dimensional figure that displays a selection of density estimates for a plausible range of smoothing parameters the boundaries of the smoothing parameter are determined by a nonparametric goodness of fit test that is based on the sample spacings for each value of the smoothing parameter our estimator is selected by choosing the normal mixture that maximizes a function of the sample spacings a point estimate is selected from this confidence set by using the method of cross validation an algorithm to find the mixing distribution that maximizes the spacings functional is presented these methods are illustrated with a data set from the astronomy literature the measurements are velocities at which galaxies in the corona borealis region are moving away from our galaxy if the galaxies are clustered the velocity density will be multimodal with clusters correspond

filterbank transceivers optimizing information rate in block transmissions over dispersive channels optimal finite impulse response fir transmit and receive filterbanks are derived for block based data transmissions over frequency selective additive gaussian noise agn channels by maximizing mutual information subject to a fixed transmit power constraint both fir and pole zero channels are considered the inherent flexibility of the proposed transceivers is exploited to derive as special cases zero forcing zf and minimum mean square error receive filterbanks the transmit filterbank converts transmission over a frequency selective fading channel affected by additive colored noise into a set of independent flat fading subchannels with uncorrelated noise samples two loading algorithms are also developed to distribute transmit power and number of bits across the usable subchannels while adhering to an upper bound on the bit error rate ber reduction of the signal to noise ratio snr margin required to satisfy the prescribed ber is achieved by coding each subchannel s bit stream the potential of the proposed transceivers is illustrated and compared to discrete multitone dmt with simulated examples

a rewriting system for categorical combinators with multiple arguments categorical combinators have been derived from the study of categorical semantics of lambda calculus and it has been found that they may be used in implementation of functional languages in this paper categorical combinators are extended so that functions with multiple arguments can be directly handled thus making them more suitable for practical computation a rewriting system named operatorname cclm beta is formulated for these combinators this system naturally corresponds to the type free lambda beta calculus the relationship between these two systems is established and as a result of this the church rosser property of operatorname cclm beta is proved a similar relationship is also established between the original operatorname ccl beta by curien and the type free lambda beta calculus with product finally the embedding theorem of operatorname cclm beta into operatorname ccl beta is shown