algebraic reflexivity of sets of bounded operators on vector valued lipschitz functions abstract in this paper we establish algebraic reflexivity properties of subsets of bounded linear operators acting on spaces of vector valued lipschitz functions we also derive a representation for the generalized bi circular projections on these spaces

optimization for training neural nets various techniques of optimizing criterion functions to train neural net classifiers are investigated these techniques include three standard deterministic techniques variable metric conjugate gradient and steepest descent and a new stochastic technique it is found that the stochastic technique is preferable on problems with large training sets and that the convergence rates of the variable metric and conjugate gradient techniques are similar

deterministic neural classification this letter presents a minimum classification error learning formulation for a single layer feedforward network slfn by approximating the nonlinear counting step function using a quadratic function the classification error rate is shown to be deterministically solvable essentially the derived solution is related to an existing weighted least squares method with class specific weights set according to the size of data set by considering the class specific weights as adjustable parameters the learning formulation extends the classification robustness of the slfn without sacrificing its intrinsic advantage of being a closed form algorithm while the method is applicable to other linear formulations our empirical results indicate slfn s effectiveness on classification generalization

fast exact multiplication by the hessian just storing the hessian h the matrix of second derivatives 2e wi wj of the error e with respect to each pair of weights of a large neural network is difficult since a common use of a large matrix like h is to compute its product with various vectors we derive a technique that directly calculates hv where v is an arbitrary vector to calculate hv we first define a differential operator rv f w r f w rv r 0 note that rv w hv and rv w v and then apply rv to the equations used to compute w the result is an exact and numerically stable procedure for computing hv which takes about as much computation and is about as local as a gradient evaluation we then apply the technique to a one pass gradient calculation algorithm backpropagation a relaxation gradient calculation algorithm recurrent backpropagation and two stochastic gradient calculation algorithms boltzmann machines and weight perturbation finally we show that this technique can be used at the heart of many iterative techniques for computing various properties of h obviating any need to calculate the full hessian

discriminant independent component analysis a conventional linear model based on negentropy maximization extracts statistically independent latent variables which may not be optimal to give a discriminant model with good classification performance in this paper a single stage linear semisupervised extraction of discriminative independent features is proposed discriminant independent component analysis dica presents a framework of linearly projecting multivariate data to a lower dimension where the features are maximally discriminant with minimal redundancy the optimization problem is formulated as the maximization of linear summation of negentropy and weighted functional measure of classification motivated by independence among extracted features fisher linear discriminant is used as the functional measure of classification experimental results show improved classification performance when dica features are used for recognition tasks in comparison to unsupervised principal component analysis and ica and supervised feature extraction techniques like linear discriminant analysis lda conditional ica and those based on information theoretic learning approaches dica features also give reduced data reconstruction error in comparison to lda and ica method based on negentropy maximization

homomorphisms between algebras of lipschitz functions with the values in function algebras abstract we give a characterization of algebra homomorphisms and isomorphisms between banach algebras of lipschitz functions with the values in unital commutative c algebras we prove that such homomorphisms are composition operators of the form and preserve some properties

algebraic reflexivity of the isometry group of some spaces of lipschitz functions abstract we show that the isometry groups of lip x d and lip x d with 0 1 for a compact metric space x d are algebraically reflexive we also prove that the sets of isometric reflections and generalized bi circular projections on such spaces are algebraically reflexive in order to achieve this we characterize generalized bi circular projections on these spaces

minimizing the discrete logarithmic energy on the sphere the role of random polynomials we prove that points in the sphere associated with roots of random polynomials via the stereographic projection are surprisingly well suited with respect to the minimal logarithmic energy on the sphere that is roots of random polynomials provide a fairly good approximation to elliptic fekete points

incremental linear discriminant analysis a fast algorithm and comparisons it has always been a challenging task to develop a fast and an efficient incremental linear discriminant analysis ilda algorithm for this purpose we conduct a new study for linear discriminant analysis lda in this paper and develop a new ilda algorithm we propose a new batch lda algorithm called lda qr lda qr is a simple and fast lda algorithm which is obtained by computing the economic qr factorization of the data matrix followed by solving a lower triangular linear system the relationship between lda qr and uncorrelated lda ulda is also revealed based on lda qr we develop a new incremental lda algorithm called ilda qr the main features of our ilda qr include that 1 it can easily handle the update from one new sample or a chunk of new samples 2 it has efficient computational complexity and space complexity and 3 it is very fast and always achieves competitive classification accuracy compared with ulda algorithm and existing ilda algorithms numerical experiments based on some real world data sets demonstrate that our ilda qr is very efficient and competitive with the state of the art ilda algorithms in terms of classification accuracy computational complexity and space complexity

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

compound rank k projections for bilinear analysis in many real world applications data are represented by matrices or high order tensors despite the promising performance the existing 2 d discriminant analysis algorithms employ a single projection model to exploit the discriminant information for projection making the model less flexible in this paper we propose a novel compound rank k projection crp algorithm for bilinear analysis the crp deals with matrices directly without transforming them into vectors and it therefore preserves the correlations within the matrix and decreases the computation complexity different from the existing 2 d discriminant analysis algorithms objective function values of crp increase monotonically in addition the crp utilizes multiple rank k projection models to enable a larger search space in which the optimal solution can be found in this way the discriminant ability is enhanced we have tested our approach on five data sets including uuim cvl pointing 04 usps and coil20 experimental results show that the performance of our proposed crp performs better than other algorithms in terms of classification accuracy

asymptotics for minimal discrete energy on the sphere we investigate the energy of arrangements of n points on the surface of the unit sphere sd in rd 1 that interact through a power law potential v 1 rs where s 0 and r is euclidean distance with ed s n denoting the minimal energy for such n point arrangements we obtain bounds valid for all n for ed s n in the cases when 0 s d and 2 d s for s d we determine the precise asymptotic behavior of ed d n as n as a corollary lower bounds are given for the separation of any pair of points in an n point minimal energy configuration when s d 2 for the unit sphere in r3 d 2 we present two conjectures concerning the asymptotic expansion of e2 s n that relate to the zeta function l s for a hexagonal lattice in the plane we prove an asymptotic upper bound that supports the first of these conjectures of related interest we derive an asymptotic formula for the partial sums of l s when 0 s 2 the divergent case

on the discriminatory power of adaptive feed forward layered networks this correspondence expands the available theoretical framework that establishes a link between discriminant analysis and adaptive feed forward layered linear output networks used as mean square classifiers this has the advantages of providing more theoretical justification for the use of these nets in pattern classification and gaining a better insight into their behavior and about their use the authors prove that under reasonable assumptions minimizing the mean square error at the network output is equivalent to minimizing the following 1 the difference between the optimum value of a familiar discriminant criterion and the value of this criterion evaluated in the space spanned 2 the outputs of the final hidden layer and 3 the difference between the values of the same discriminant criterion evaluated in desired output and actual output subspaces the authors also illustrate under specific constraints how to solve the following problem given a feature extraction criterion how the target coding scheme can be selected such that this criterion is maximized at the output of the network final hidden layer other properties for these networks are explored

dictionary learning based subspace structure identification in spectral clustering in this paper we study dictionary learning dl approach to identify the representation of low dimensional subspaces from high dimensional and nonnegative data such representation can be used to provide an affinity matrix among different subspaces for data clustering the main contribution of this paper is to consider both nonnegativity and sparsity constraints together in dl such that data can be represented effectively by nonnegative and sparse coding coefficients and nonnegative dictionary bases in the algorithm we employ the proximal point technique for the resulting dl and sparsity optimization problem we make use of coding coefficients to perform spectral clustering sc for data partitioning extensive experiments on real world high dimensional and nonnegative data sets including text microarray and image data demonstrate that the proposed method can discover their subspace structures experimental results also show that our algorithm is computationally efficient and effective for obtaining high sc performance and interpreting the clustering results compared with the other testing methods

simple exponential family pca principal component analysis pca is a widely used model for dimensionality reduction in this paper we address the problem of determining the intrinsic dimensionality of a general type data population by selecting the number of principal components for a generalized pca model in particular we propose a generalized bayesian pca model which deals with general type data by employing exponential family distributions model selection is realized by empirical bayesian inference of the model we name the model as simple exponential family pca sepca since it embraces both the principal of using a simple model for data representation and the practice of using a simplified computational procedure for the inference our analysis shows that the empirical bayesian inference in sepca formally realizes an intuitive criterion for pca model selection a preserved principal component must sufficiently correlate to data variance that is uncorrelated to the other principal components experiments on synthetic and real data sets demonstrate effectiveness of sepca and exemplify its characteristics for model selection

feature extraction with deep neural networks by a generalized discriminant analysis we present an approach to feature extraction that is a generalization of the classical linear discriminant analysis lda on the basis of deep neural networks dnns as for lda discriminative features generated from independent gaussian class conditionals are assumed this modeling has the advantages that the intrinsic dimensionality of the feature space is bounded by the number of classes and that the optimal discriminant function is linear unfortunately linear transformations are insufficient to extract optimal discriminative features from arbitrarily distributed raw measurements the generalized discriminant analysis gerda proposed in this paper uses nonlinear transformations that are learnt by dnns in a semisupervised fashion we show that the feature extraction based on our approach displays excellent performance on real world recognition and detection tasks such as handwritten digit recognition and face detection in a series of experiments we evaluate gerda features with respect to dimensionality reduction visualization classification and detection moreover we show that gerda dnns can preprocess truly high dimensional input data to low dimensional representations that facilitate accurate predictions even if simple linear predictors or measures of similarity are used

deep adaptive feature embedding with local sample distributions for person re identification person re identification re id aims to match pedestrians observed by disjoint camera views it attracts increasing attention in computer vision due to its importance to surveillance systems to combat the major challenge of cross view visual variations deep embedding approaches are proposed by learning a compact feature space from images such that the euclidean distances correspond to their cross view similarity metric however the global euclidean distance cannot faithfully characterize the ideal similarity in a complex visual feature space because features of pedestrian images exhibit unknown distributions due to large variations in poses illumination and occlusion moreover intra personal training samples within a local range which are robust to guide deep embedding against uncontrolled variations cannot be captured by a global euclidean distance in this paper we study the problem of person re id by proposing a novel sampling to mine suitable positives i e intra class within a local range to improve the deep embedding in the context of large intra class variations our method is capable of learning a deep similarity metric adaptive to local sample structure by minimizing each sample s local distances while propagating through the relationship between samples to attain the whole intra class minimization to this end a novel objective function is proposed to jointly optimize similarity metric learning local positive mining and robust deep feature embedding this attains local discriminations by selecting local ranged positive samples and the learned features are robust to dramatic intra class variations experiments on benchmarks show state of the art results achieved by our method c 2017 elsevier ltd all rights reserved

semisupervised generalized discriminant analysis generalized discriminant analysis gda is a commonly used method for dimensionality reduction in its general form it seeks a nonlinear projection that simultaneously maximizes the between class dissimilarity and minimizes the within class dissimilarity to increase class separability in real world applications where labeled data are scarce gda may not work very well however unlabeled data are often available in large quantities at very low cost in this paper we propose a novel gda algorithm which is abbreviated as semisupervised generalized discriminant analysis ssgda we utilize unlabeled data to maximize an optimality criterion of gda and formulate the problem as an optimization problem that is solved using the constrained concave convex procedure the optimization procedure leads to estimation of the class labels for the unlabeled data we propose a novel confidence measure and a method for selecting those unlabeled data points whose labels are estimated with high confidence the selected unlabeled data can then be used to augment the original labeled dataset for performing gda we also propose a variant of ssgda called m ssgda which adopts the manifold assumption to utilize the unlabeled data extensive experiments on many benchmark datasets demonstrate the effectiveness of our proposed methods

smale s 17th problem average polynomial time to compute affine and projective solutions smale s 17th problem asks can a zero of n complex polynomial equations in n unknowns be found approximately on the average in polynomial time with a uniform algorithm we give a positive answer to this question namely we describe a uniform probabilistic algorithm that computes an approximate zero of systems of polynomial equations f cn cn performing a number of arithmetic operations which is polynomial in the size of the input on the average

deep linear discriminant analysis on fisher networks person re identification is to seek a correct match for a person of interest across different camera views among a large number of impostors it typically involves two procedures of non linear feature extractions against dramatic appearance changes and subsequent discriminative analysis in order to reduce intra personal variations while enlarging inter personal differences in this paper we introduce a hybrid deep architecture which combines fisher vectors and deep neural networks to learn non linear transformations of pedestrian images to a deep space where data can be linearly separable the proposed method starts from fisher vector encoding which computes a sequence of local feature extraction aggregation and encoding the resulting fisher vector output are fed into stacked supervised layer to seek non linear transformation into a deep space on top of the deep neural network linear discriminant analysis lda is reinforced such that linearly separable latent representations can be learned in an end to end fashion by optimizing an objective function modified from lda the network is enforced to produce feature distributions which have a low variance within the same class and high variance between classes the objective is essentially derived from the general lda eigenvalue problem and allows to train the network with stochastic gradient descent and back propagate lda gradients to compute gaussian mixture model gmm gradients in fisher vector encoding for empirical evaluations we test our approach on four benchmark data sets in person re identification viper 1 cuhk03 2 cuhk01 3 and market 1501 4 extensive experiments on these benchmarks show that our method can achieve state of the art results highlightsa hybrid architecture that combines fisher vectors and deep neural networks an end to end training with linear discriminant analysis as objective deep features are linearly separable and class separability is maximally preserved

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

flexible manifold embedding a framework for semi supervised and unsupervised dimension reduction we propose a unified manifold learning framework for semi supervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points for semi supervised dimension reduction we aim to find the optimal prediction labels f for all the training samples x the linear regression function h x and the regression residue f0 f h x simultaneously our new objective function integrates two terms related to label fitness and manifold smoothness as well as a flexible penalty term defined on the residue f0 our semi supervised learning framework referred to as flexible manifold embedding fme can effectively utilize label information from labeled data as well as a manifold structure from both labeled and unlabeled data by modeling the mismatch between h x and f we show that fme relaxes the hard linear constraint f h x in manifold regularization mr making it better cope with the data sampled from a nonlinear manifold in addition we propose a simplified version referred to as fme u for unsupervised dimension reduction we also show that our proposed framework provides a unified view to explain and understand many semi supervised supervised and unsupervised dimension reduction techniques comprehensive experiments on several benchmark databases demonstrate the significant improvement over existing dimension reduction algorithms

random riesz energies on compact kahler manifolds the expected riesz energies e n h es of the zero sets of systems of independent gaussian random polynomials are determined asymptotically as the degree n in all dimensions and codimensions the asymptotics are proved for sections of any positive line bundle over any compact kahler manifold

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

convolutional sparse coding for trajectory reconstruction trajectory basis non rigid structure from motion nrsfm refers to the process of reconstructing the 3d trajectory of each point of a non rigid object from just their 2d projected trajectories reconstruction relies on two factors i the condition of the composed camera trajectory basis matrix and ii whether the trajectory basis has enough degrees of freedom to model the 3d point trajectory these two factors are inherently conflicting employing a trajectory basis with small capacity has the positive characteristic of reducing the likelihood of an ill conditioned system when composed with the camera during reconstruction however this has the negative characteristic of increasing the likelihood that the basis will not be able to fully model the object s true 3d point trajectories in this paper we draw upon a well known result centering around the reduced isometry property rip condition for sparse signal reconstruction rip allow us to relax the requirement that the full trajectory basis composed with the camera matrix must be well conditioned further we propose a strategy for learning an over complete basis using convolutional sparse coding from naturally occurring point trajectory corpora to increase the likelihood that the rip condition holds for a broad class of point trajectories and camera motions finally we propose an ell 1 inspired objective for trajectory reconstruction that is able to adaptively select the smallest sub matrix from an over complete trajectory basis that balances i and ii we present more practical 3d reconstruction results compared to current state of the art in trajectory basis nrsfm

qmc designs optimal order quasi monte carlo integration schemes on the sphere we study equal weight numerical integration or quasi monte carlo qmc rules for functions in a sobolev space hs sd with smoothness parameter s d 2 defined over the unit sphere sd in rd 1 focusing on n point configurations that achieve optimal order qmc error bounds as is the case for efficient spherical designs we are led to introduce the concept of qmc designs these are sequences of n point configurations xn on s d such that the worst case error satisfies sup f h s f hs 1 1 n x xn f x sd f x d d x o n s d n with an implied constant that depends on the hs sd norm but is independent of n here d is the normalized surface measure on s d we provide methods for generation and numerical testing of qmc designs an essential tool is an expression for the worst case error in terms of a reproducing kernel for the space hs sd with s d 2 as a consequence of this and a recent result of bondarenko et al on the existence of spherical designs with appropriate number of points we show that minimizers of the n point energy for this kernel form a sequence of qmc designs for hs sd furthermore without appealing to the bondarenko et al result we prove that point sets that maximize the sum of suitable powers of the euclidean distance between pairs of points form a sequence of qmc designs for hs sd with s in the interval d 2 d 2 1 for such spaces there exist reproducing kernels with simple closed forms that are useful for numerical testing of optimal order quasi monte carlo integration numerical experiments suggest that many familiar sequences of point sets on the sphere equal area points spiral points minimal coulomb or logarithmic energy points and fekete points are qmc designs for appropriate values of s for comparison purposes we show that configurations of random points that are independently and uniformly distributed on the sphere do not constitute qmc designs for any s d 2 if xn is a sequence of qmc designs for h s sd we prove that it is also a sequence of qmc designs for hs sd for all s d 2 s this leads to the question of determining the supremum of such s here called the qmc strength of the sequence for which we provide estimates based on computations for the aforementioned sequences received by the editor august 15 2012 and in revised form february 26 2013 2010 mathematics subject classification primary 65d30 65d32 secondary 11k38 41a55

a novel radial basis function neural network for discriminant analysis a novel radial basis function neural network for discriminant analysis is presented in this paper in contrast to many other researches this work focuses on the exploitation of the weight structure of radial basis function neural networks using the bayesian method it is expected that the performance of a radial basis function neural network with a well explored weight structure can be improved as the weight structure of a radial basis function neural network is commonly unknown the bayesian method is therefore used in this paper to study this a priori structure two weight structures are investigated in this study i e a single gaussian structure and a two gaussian structure an expectation maximization learning algorithm is used to estimate the weights the simulation results showed that the proposed radial basis function neural network with a weight structure of two gaussians outperformed the other algorithms

nonstationary source separation using sequential and variational bayesian learning independent component analysis ica is a popular approach for blind source separation where the mixing process is assumed to be unchanged with a fixed set of stationary source signals however the mixing system and source signals are nonstationary in real world applications e g the source signals may abruptly appear or disappear the sources may be replaced by new ones or even moving by time this paper presents an online learning algorithm for the gaussian process gp and establishes a separation procedure in the presence of nonstationary and temporally correlated mixing coefficients and source signals in this procedure we capture the evolved statistics from sequential signals according to online bayesian learning the activity of nonstationary sources is reflected by an automatic relevance determination which is incrementally estimated at each frame and continuously propagated to the next frame we employ the gp to characterize the temporal structures of time varying mixing coefficients and source signals a variational bayesian inference is developed to approximate the true posterior for estimating the nonstationary ica parameters and for characterizing the activity of latent sources the differences between this ica method and the sequential monte carlo ica are illustrated in the experiments the proposed algorithm outperforms the other ica methods for the separation of audio signals in the presence of different nonstationary scenarios

image classification with the fisher vector theory and practice a standard approach to describe an image for classification and retrieval purposes is to extract a set of local patch descriptors encode them into a high dimensional vector and pool them into an image level signature the most common patch encoding strategy consists in quantizing the local descriptors into a finite set of prototypical elements this leads to the popular bag of visual words representation in this work we propose to use the fisher kernel framework as an alternative patch encoding strategy we describe patches by their deviation from an universal generative gaussian mixture model this representation which we call fisher vector has many advantages it is efficient to compute it leads to excellent results even with efficient linear classifiers and it can be compressed with a minimal loss of accuracy using product quantization we report experimental results on five standard datasets pascal voc 2007 caltech 256 sun 397 ilsvrc 2010 and imagenet10k with up to 9m images and 10k classes showing that the fv framework is a state of the art patch encoding technique

algebraic reflexivity of isometry groups of algebras of lipschitz maps abstract we study groups of surjective linear isometries on banach algebras of lipschitz maps with values in some unital c algebras in this paper these spaces are endowed with the sum norm for the case where the c algebras are commutative whose groups of all surjective isometries are algebraically reflexive we prove that the group of all surjective isometries on the corresponding banach algebra of lipschitz maps are algebraically reflexive we also prove that the group of unital surjective isometries between matrix valued lipschitz algebras are reflexive

universal approximation using incremental constructive feedforward networks with random hidden nodes according to conventional neural network theories single hidden layer feedforward networks slfns with additive or radial basis function rbf hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable however as observed in most neural network implementations tuning all the parameters of the networks may cause learning complicated and inefficient and it may be difficult to train networks with nondifferential activation functions such as threshold networks unlike conventional neural network theories this paper proves in an incremental constructive method that in order to let slfns work as universal approximators one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer in such slfns implementations the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g r r and the activation functions for rbf nodes can be any integrable piecewise continuous functions g r r and rg x dx 0 the proposed incremental method is efficient not only for sflns with continuous including nondifferentiable activation functions but also for slfns with piecewise continuous such as threshold activation functions compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

generalization performance of fisher linear discriminant based on markov sampling fisher linear discriminant fld is a well known method for dimensionality reduction and classification that projects high dimensional data onto a low dimensional space where the data achieves maximum class separability the previous works describing the generalization ability of fld have usually been based on the assumption of independent and identically distributed i i d samples in this paper we go far beyond this classical framework by studying the generalization ability of fld based on markov sampling we first establish the bounds on the generalization performance of fld based on uniformly ergodic markov chain u e m c samples and prove that fld based on u e m c samples is consistent by following the enlightening idea from markov chain monto carlo methods we also introduce a markov sampling algorithm for fld to generate u e m c samples from a given data of finite size through simulation studies and numerical studies on benchmark repository using fld we find that fld based on u e m c samples generated by markov sampling can provide smaller misclassification rates compared to i i d samples

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

computational cost of the fekete problem i the forces method on the 2 sphere here we study the computational complexity of the fekete point problem namely we give an exhaustive description of the main properties of an algorithm for the minimization of the logarithmic potential energy on the 2 sphere and we characterize the probability distribution of the cost of the different minima in particular we show that a local minimum can be found with an average cost of about o n 2 8

a rank one update algorithm for fast solving kernel foley sammon optimal discriminant vectors discriminant analysis plays an important role in statistical pattern recognition a popular method is the foley sammon optimal discriminant vectors fsodvs method which aims to find an optimal set of discriminant vectors that maximize the fisher discriminant criterion under the orthogonal constraint the fsodvs method outperforms the classic fisher linear discriminant analysis flda method in the sense that it can solve more discriminant vectors for recognition kernel foley sammon optimal discriminant vectors kfsodvs is a nonlinear extension of fsodvs via the kernel trick however the current kfsodvs algorithm may suffer from the heavy computation problem since it involves computing the inverse of matrices when solving each discriminant vector resulting in a cubic complexity for each discriminant vector this is costly when the number of discriminant vectors to be computed is large in this paper we propose a fast algorithm for solving the kfsodvs which is based on rank one update rou of the eigensytems it only requires a square complexity for each discriminant vector moreover we also generalize our method to efficiently solve a family of optimally constrained generalized rayleigh quotient ocgrq problems which include many existing dimensionality reduction techniques we conduct extensive experiments on several real data sets to demonstrate the effectiveness of the proposed algorithms

short note efficient and accurate rotation of finite spherical harmonics expansions spherical harmonics are employed in a wide range of applications in computational science and physics and many of them require the rotation of functions we present an efficient and accurate algorithm for the rotation of finite spherical harmonics expansions exploiting the pointwise action of the rotation group on functions on the sphere we obtain the spherical harmonics expansion of a rotated signal from function values at rotated sampling points the number of sampling points and their location permits one to balance performance and accuracy making our technique well suited for a wide range of applications numerical experiments comparing different sampling schemes and various techniques from the literature are presented making this the first thorough evaluation of spherical harmonics rotation algorithms

isometries between function spaces surjective isometries between some classical function spaces are investigated we give a simple technical scheme which verifies whether any such isometry is given by a homeomorphism between corresponding hausdorff compact spaces in particular the answer is positive for the c x ac 0 1 lip0 x and lip0 x spaces provided with various natural norms

a real quaternion spherical ensemble of random matrices one can identify a tripartite classification of random matrix ensembles into geometrical universality classes corresponding to the plane the sphere and the anti sphere the plane is identified with ginibre type iid matrices and the anti sphere with truncations of unitary matrices this paper focusses on an ensemble corresponding to the sphere matrices of the form y a 1 b where a and b are independent n n matrices with iid standard gaussian real quaternion entries by applying techniques similar to those used for the analogous complex and real spherical ensembles the eigenvalue joint probability density function and correlation functions are calculated this completes the exploration of spherical matrices using the traditional dyson indices 1 2 4

reflexivity algebraic reflexivity and linear interpolation introduction this paper was motivated by a discovery that most finite dimensional subalgebras of b h for h an infinite dimensional hilbert space are reflexive with the only obstructions to reflexivity being finite rank considerations proofs section 2 do not depend on multiplicative structure nor on topology so extend to linear subspaces of transformations in an abstract setting abstract reflexivity has been studied in 3 4 7 primarily for singly generated algebras reflexivity properties can be interpreted as linear interpolation properties and we shall adopt this point of view the results of section 2 extend in section 3 to algebraic reflexivity counterparts for countably generated algebraically linear subspaces of bounded linear transformations acting on a banach space as consequences in section 4 we obtain generalizations of two single operator results one is a multivariate version of the result due to kaplansky 10 theorem 15 that a bounded locally algebraic operator acting on a banach space is algebraic the other is a multivariate version of the result due to douglas and foias 5 for hilbert space and to hadwin 7 for a general banach space extending work of fillmore 6 that a bounded non algebraic operator acting on a banach space is topologically algebraically reflexive in section 5 we give an application kindly suggested by d hadwin concerning joint strong similarity orbits of n tupules of operators we wish to thank e azoff k davidson j erdos d hadwin j kraus c lance and a sourour for conversations and some correspon

reidentification by relative distance comparison matching people across nonoverlapping camera views at different locations and different times known as person reidentification is both a hard and important problem for associating behavior of people observed in a large distributed space over a prolonged period of time person reidentification is fundamentally challenging because of the large visual appearance changes caused by variations in view angle lighting background clutter and occlusion to address these challenges most previous approaches aim to model and extract distinctive and reliable visual features however seeking an optimal and robust similarity measure that quantifies a wide range of features against realistic viewing conditions from a distance is still an open and unsolved problem for person reidentification in this paper we formulate person reidentification as a relative distance comparison rdc learning problem in order to learn the optimal similarity measure between a pair of person images this approach avoids treating all features indiscriminately and does not assume the existence of some universally distinctive and reliable features to that end a novel relative distance comparison model is introduced the model is formulated to maximize the likelihood of a pair of true matches having a relatively smaller distance than that of a wrong match pair in a soft discriminant manner moreover in order to maintain the tractability of the model in large scale learning we further develop an ensemble rdc model extensive experiments on three publicly available benchmarking datasets are carried out to demonstrate the clear superiority of the proposed rdc models over related popular person reidentification techniques the results also show that the new rdc models are more robust against visual appearance changes and less susceptible to model overfitting compared to other related existing models

evaluation of weighted fisher criteria for large category dimensionality reduction in application to chinese handwriting recognition to improve the class separability of fisher linear discriminant analysis fda for large category problems we investigate the weighted fisher criterion wfc by integrating weighting functions for dimensionality reduction the objective of wfc is to maximize the sum of weighted distances of all class pairs by setting larger weights for the most confusable classes wfc can improve the class separation while the solution remains an eigen decomposition problem we evaluate five weighting functions in three different weighting spaces in a typical large category problem of handwritten chinese character recognition the weighting functions include four based on existing methods namely fda approximate pairwise accuracy criterion apac power function pow confused distance maximization cdm and a new one based on k nearest neighbors knn all the weighting functions can be calculated in the original feature space low dimensional space or fractional space our experiments on a 3 755 class chinese handwriting database demonstrate that wfc can improve the classification accuracy significantly compared to fda among the weighting functions the knn method in the original space is the most competitive model which achieves significantly higher classification accuracy and has a low computational complexity to further improve the performance we propose a nonparametric extension of the knn method from the class level to the sample level the sample level knn sknn method is shown to outperform significantly other methods in chinese handwriting recognition such as the locally linear discriminant analysis llda neighbor class linear discriminant analysis nclda and heteroscedastic linear discriminant analysis hlda