sparse finite elements for elliptic problems with stochastic loading we formulate elliptic boundary value problems with stochastic loading in a bounded domain d d we show well posedness of the problem in stochastic sobolev spaces and we derive a deterministic elliptic pde in d d for the spatial correlation of the random solution we show well posedness and regularity results for this pde in a scale of weighted sobolev spaces with mixed highest order derivatives discretization with sparse tensor products of any hierarchic finite element fe spaces in d yields optimal asymptotic rates of convergence for the spatial correlation even in the presence of singularities or for spatially completely uncorrelated data multilevel preconditioning in d d allows iterative solution of the discrete equation for the correlation kernel in essentially the same complexity as the solution of the mean field equation

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

a penalized neumann control approach for solving an optimal dirichlet control problem for the navier stokes equations we introduce a penalized neumann boundary control approach for solving an optimal dirichlet boundary control problem associated with the two or three dimensional steady state navier stokes equations we prove the convergence of the solutions of the penalized neumann control problem the suboptimality of the limit and the optimality of the limit under further restrictions on the data we describe the numerical algorithm for solving the penalized neumann control problem and report some numerical results

locality regularized linear regression discriminant analysis for feature extraction abstract locality regularized linear regression classification llrc is an effective classifier that shows great potential for face recognition however the original feature space cannot guarantee the classification efficiency of llrc to alleviate this problem we propose a novel dimensionality reduction method called locality regularized linear regression discriminant analysis llrda for feature extraction the proposed llrda is developed according to the decision rule of llrc and seeks to generate a subspace that is discriminant for llrc specifically the intra class and inter class local reconstruction scatters are first defined to characterize the compactness and separability of samples respectively then the objective function for llrda is derived by maximizing the inter class local reconstruction scatter and simultaneously minimizing the intra class local reconstruction scatter extensive experimental results on cmu pie orl feret and yale b face databases validate the effectiveness of our proposed method

linear reconstruction measure steered nearest neighbor classification framework the linear reconstruction measure lrm which determines the nearest neighbors of the query sample in all known training samples by sorting the minimum l 2 norm error linear reconstruction coefficients is introduced in this paper the intuitive interpretation and mathematical proofs are presented to reveal the efficient working mechanism of lrm through analyzing the physical meaning of coefficients and regularization items we find that lrm provides more useful information and advantages than the conventional similarity measure model which calculates the distance between two entities i e conventional point to point c ptp inspired by the advantages of lrm the linear reconstruction measure steered nearest neighbor classification framework lrm nncf is designed with eight classifiers according to different decision rules and models of lrm evaluation on several face databases and the experimental results demonstrate that these proposed classifiers can achieve greater performance than the c ptp based 1 nns and competitive recognition accuracy and robustness compared with the state of the art classifiers

solving elliptic boundary value problems with uncertain coefficients by the finite element method the stochastic formulation this work studies a linear elliptic problem with uncertainty the introduction gives a survey of different formulations of the uncertainty and resulting numerical approximations the major emphasis

comparison of approaches for random pde optimization problems based on different matching functionals in this article we consider an optimal control problem for an elliptic partial differential equation with random inputs to determine an applicable deterministic control f x we consider the four cases which we compare for efficiency and feasibility we prove the existence of optimal states adjoint states and optimality conditions for each cases we also derive the optimality systems for the four cases the optimality system is then discretized by a standard finite element method and sparse grid collocation method for physical space and probability space respectively the numerical experiments are performed for their efficiency and feasibility

locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image we present a novel method of nonlinear discriminant analysis involving a set of locally linear transformations called locally linear discriminant analysis llda the underlying idea is that global nonlinear data structures are locally linear and local structures can be linearly aligned input vectors are projected into each local feature space by linear transformations found to yield locally linearly transformed classes that maximize the between class covariance while minimizing the within class covariance in face recognition linear discriminant analysis lida has been widely adopted owing to its efficiency but it does not capture nonlinear manifolds of faces which exhibit pose variations conventional nonlinear classification methods based on kernels such as generalized discriminant analysis gda and support vector machine svm have been developed to overcome the shortcomings of the linear method but they have the drawback of high computational cost of classification and overfitting our method is for multiclass nonlinear discrimination and it is computationally highly efficient as compared to gda the method does not suffer from overfitting by virtue of the linear base structure of the solution a novel gradient based learning algorithm is proposed for finding the optimal set of local linear bases the optimization does not exhibit a local maxima problem the transformation functions facilitate robust face recognition in a low dimensional subspace under pose variations using a single model image the classification results are given for both synthetic and real face data

why can lda be performed in pca transformed space abstract pca plus lda is a popular framework for linear discriminant analysis lda in high dimensional and singular case in this paper we focus on building a theoretical foundation for this framework moreover we point out the weakness of the previous lda based methods and suggest a complete pca plus lda algorithm experimental results on orl face image database indicate that the proposed method is more effective than the previous ones

fractional order embedding canonical correlation analysis and its applications to multi view dimensionality reduction and recognition due to the noise disturbance and limited number of training samples within set and between set sample covariance matrices in canonical correlation analysis cca usually deviate from the true ones in this paper we re estimate within set and between set covariance matrices to reduce the negative effect of this deviation specifically we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices and then construct fractional order within set and between set scatter matrices which can obviously alleviate the problem of the deviation on this basis a new approach is proposed to reduce the dimensionality of multi view data for classification tasks called fractional order embedding canonical correlation analysis fecca the proposed method is evaluated on various handwritten numeral face and object recognition problems extensive experimental results on the cenparmi uci at t ar and coil 20 databases show that fecca is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy moreover its improvements for recognition rates are statistically significant on most cases below the significance level 0 05 a new method namely fecca is presented for multi view dimensionality reduction fractional order within set and between set scatter matrices are constructed by sample spectrum modeling the extracted features by fecca have strong discriminant power for recognition experimental results show fecca has better recognition rates than the existing joint feature extraction methods

discriminative common vectors for face recognition in face recognition tasks the dimension of the sample space is typically larger than the number of the samples in the training set as a consequence the within class scatter matrix is singular and the linear discriminant analysis lda method cannot be applied directly this problem is known as the small sample size problem in this paper we propose a new face recognition method called the discriminative common vector method based on a variation of fisher s linear discriminant analysis for the small sample size case two different algorithms are given to extract the discriminative common vectors representing each person in the training set of the face database one algorithm uses the within class scatter matrix of the samples in the training set while the other uses the subspace methods and the gram schmidt orthogonalization procedure to obtain the discriminative common vectors then the discriminative common vectors are used for classification of new faces the proposed method yields an optimal solution for maximizing the modified fisher s linear discriminant criterion given in the paper our test results show that the discriminative common vector method is superior to other methods in terms of recognition accuracy efficiency and numerical stability

locally adaptive metric nearest neighbor classification nearest neighbor classification assumes locally constant class conditional probabilities this assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality severe bias can be introduced under these conditions when using the nearest neighbor rule we propose a locally adaptive nearest neighbor classification method to try to minimize bias we use a chi squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones as a result the class conditional probabilities are smoother in the modified neighborhoods whereby better classification performance can be achieved the efficacy of our method is validated and compared against other techniques using both simulated and real world data

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

an optimization based domain decomposition method for pdes with random inputs an optimization based domain decomposition method for stochastic elliptic partial differential equations is presented the main idea of the method is a constrained optimization problem for which the minimization of an appropriate functional forces the solutions on the two subdomains to agree on the interface the constraints are the stochastic partial differential equations the existence of optimal solutions for the stochastic optimal control problem is shown as is the convergence to the exact solution of the given problem we prove the existence of a lagrange multiplier and derive an optimality system from which solutions of the domain decomposition problem may be determined finite element approximations to the solutions of the optimality system are defined and analyzed with respect to both spatial and random parameter spaces then the results of some numerical experiments are given to confirm theoretical error estimate results

optimal control with stochastic pde constraints and uncertain controls abstract the optimal control of problems that are constrained by partial differential equations with uncertainties and with uncertain controls is addressed the lagrangian that defines the problem is postulated in terms of stochastic functions with the control function possibly decomposed into an unknown deterministic component and a known zero mean stochastic component the extra freedom provided by the stochastic dimension in defining cost functionals is explored demonstrating the scope for controlling statistical aspects of the system response one shot stochastic finite element methods are used to find approximate solutions to control problems it is shown that applying the stochastic collocation finite element method to the formulated problem leads to a coupling between stochastic collocation points when a deterministic optimal control is considered or when moments are included in the cost functional thereby forgoing the primary advantage of the collocation method over the stochastic galerkin method for the considered problem the application of the presented methods is demonstrated through a number of numerical examples the presented framework is sufficiently general to also consider a class of inverse problems and numerical examples of this type are also presented

discriminative histograms of local dominant orientation d hldo for biometric image feature extraction this paper presents a simple and robust method namely discriminative histograms of local dominant orientation d hldo for biometric image feature extraction in d hldo the local dominant orientation map and the corresponding relative energy map are obtained by applying the singular value decomposition svd to the collected gradient vectors over a local patch the dominant orientation map and the relative energy map are then used to construct the concatenated histogram features local mean based nearest neighbor discriminant analysis lm nnda is finally employed to reduce the redundancy information and get the low dimensional and discriminative features the proposed method is applied to face finger knuckle print and palm biometrics and is examined using the ar cmu pie and frgcv2 0 face image databases the polyu palmprint database and the polyu finger knuckle print database experimental results demonstrate the effectiveness of the proposed d hldo method

discriminant adaptive nearest neighbor classification nearest neighbour classification expects the class conditional probabilities to be locally constant and suffers from bias in high dimensions we propose a locally adaptive form of nearest neighbour classification to try to ameliorate this curse of dimensionality we use a local linear discriminant analysis to estimate an effective metric for computing neighbourhoods we determine the local decision boundaries from centroid information and then shrink neighbourhoods in directions orthogonal to these local decision boundaries and elongate them parallel to the boundaries thereafter any neighbourhood based classifier can be employed using the modified neighbourhoods the posterior probabilities tend to be more homogeneous in the modified neighbourhoods we also propose a method for global dimension reduction that combines local dimension information in a number of examples the methods demonstrate the potential for substantial improvements over nearest neighbour classification

solution of stochastic partial differential equations using galerkin finite element techniques this paper presents a framework for the construction of galerkin approximations of elliptic boundary value problems with stochastic input data a variational formulation is developed which allows among others numerical treatment by the finite element method a theory of a posteriori error estimation and corresponding adaptive approaches based on practical experience can be utilized the paper develops a foundation for treating stochastic partial differential equations pdes which can be further developed in many directions

rectified nearest feature line segment for pattern classification this paper points out and analyzes the advantages and drawbacks of the nearest feature line nfl classifier to overcome the shortcomings a new feature subspace with two simple and effective improvements is built to represent each class the proposed method termed rectified nearest feature line segment rnfls is shown to possess a novel property of concentration as a result of the added line segments features which significantly enhances the classification ability another remarkable merit is that rnfls is applicable to complex tasks such as the two spiral distribution which the original nfl cannot deal with properly finally experimental comparisons with nfl nn nearest neighbor k nn and nnl nearest neighbor line using both artificial and real world data sets demonstrate that rnfls offers the best performance

from classifiers to discriminators a nearest neighbor rule induced discriminant analysis the current discriminant analysis method design is generally independent of classifiers thus the connection between discriminant analysis methods and classifiers is loose this paper provides a way to design discriminant analysis methods that are bound with classifiers we begin with a local mean based nearest neighbor lm nn classifier and use its decision rule to supervise the design of a discriminator therefore the derived discriminator called local mean based nearest neighbor discriminant analysis lm nnda matches the lm nn classifier optimally in theory in contrast to that lm nnda is a nn classifier induced discriminant analysis method we further show that the classical fisher linear discriminant analysis flda is a minimum distance classifier i e nearest class mean classifier induced discriminant analysis method the proposed lm nnda method is evaluated using the cenparmi handwritten numeral database the nust603 handwritten chinese character database the eth80 object category database and the feret face image database the experimental results demonstrate the performance advantage of lm nnda over other feature extraction methods with respect to the lm nn or nn classifier

analysis and finite element approximation of optimal control problems for the stationary navier stokes equations with distributed and neumann controls we examine certain analytic and numerical aspects of optimal control problems for the stationary navier stokes equations the controls considered may be of either the distributed or neumann type the functionals minimized are either the viscous dissipation or the l4 distance of candidate flows to some desired flow we show the existence of optimal solutions and justify the use of lagrange multiplier techniques to derive a system of partial differential equations from which optimal solutions may be deduced we study the regularity of solutions of this system then we consider the approximation by finite element methods of solutions of the optimality system and derive optimal error estimates

optimal approximation to a class of nonlinear evolution equations abstract in this paper we have developed and discussed a generalized difference method gdm for the approximation of a class of nonlinear evolution equations with integral items in one space dimension the main idea of the paper is to introduce a generalized volterra projection operator and we design a generalized difference formulation with a new approximate value of initial function which is of special importance for the error analysis then some theories of the generalized volterra projection operator are analyzed finally we deduce the optimal error estimates moreover some conclusions are obtained and our further research work is proposed

on solving elliptic stochastic partial differential equations abstract a model elliptic boundary value problem of second order with stochastic coefficients described by the karhunen loeve expansion is addressed this problem is transformed into an equivalent deterministic one the perturbation method and the method of successive approximations is analyzed rigorous error estimates in the framework of sobolev spaces are given

a recursive description of pro p galois groups abstract in this note we study a modified version of the elementary type conjecture for pro p galois groups to be precise let g p f be the galois group of the maximal galois p extension of a field f containing a primitive p th root of unity under some natural assumptions concerning valuation rings of f and also orderings of f when p 2 we prove that g p f can be obtained from suitable closed subgroups using a finite number of free pro p products and semidirect products

approximating infinity dimensional stochastic darcy s equations without uniform ellipticity we consider a stochastic darcy s pressure equation whose coefficient is generated by a white noise process on a hilbert space employing the ordinary rather than the wick product a weak form of this equation involves different spaces for the solution and test functions and we establish a continuous inf sup condition and well posedness of the problem we generalize the numerical approximations proposed in benth and theting stochastic anal appl 20 2002 pp 1191 1223 for wick stochastic partial differential equations to the ordinary product stochastic pressure equation we establish discrete inf sup conditions and provide a priori error estimates for a wide class of norms the proposed numerical approximation is based on wiener chaos finite element methods and yields a positive definite symmetric linear system we also improve and generalize the approximation results of benth and gjerde stochastics stochastics rep 63 1998 pp 313 326 and cao stochastics 78 2006 pp 179 187 when a generalized process is truncated by a finite wiener chaos expansion finally we present numerical experiments to validate the results

face recognition using kernel direct discriminant analysis algorithms techniques that can introduce low dimensional feature representation with enhanced discriminatory power is of paramount importance in face recognition fr systems it is well known that the distribution of face images under a perceivable variation in viewpoint illumination or facial expression is highly nonlinear and complex it is therefore not surprising that linear techniques such as those based on principle component analysis pca or linear discriminant analysis lda cannot provide reliable and robust solutions to those fr problems with complex face variations in this paper we propose a kernel machine based discriminant analysis method which deals with the nonlinearity of the face patterns distribution the proposed method also effectively solves the so called small sample size sss problem which exists in most fr tasks the new algorithm has been tested in terms of classification error rate performance on the multiview umist face database results indicate that the proposed methodology is able to achieve excellent performance with only a very small set of features being used and its error rate is approximately 34 and 48 of those of two other commonly used kernel fr approaches the kernel pca kpca and the generalized discriminant analysis gda respectively

kpca plus lda a complete kernel fisher discriminant framework for feature extraction and recognition this paper examines the theory of kernel fisher discriminant analysis kfd in a hilbert space and develops a two phase kfd framework i e kernel principal component analysis kpca plus fisher linear discriminant analysis lda this framework provides novel insights into the nature of kfd based on this framework the authors propose a complete kernel fisher discriminant analysis ckfd algorithm ckfd can be used to carry out discriminant analysis in double discriminant subspaces the fact that it can make full use of two kinds of discriminant information regular and irregular makes ckfd a more powerful discriminator the proposed algorithm was tested and evaluated using the feret face database and the cenparmi handwritten numeral database the experimental results show that ckfd outperforms other kfd algorithms

bayes optimality in linear discriminant analysis we present an algorithm that provides the one dimensional subspace where the bayes error is minimized for the c class problem with homoscedastic gaussian distributions our main result shows that the set of possible one dimensional spaces v for which the order of the projected class means is identical defines a convex region with associated convex bayes error function g v this allows for the minimization of the error function using standard convex optimization algorithms our algorithm is then extended to the minimization of the bayes error in the more general case of heteroscedastic distributions this is done by means of an appropriate kernel mapping function this result is further extended to obtain the d dimensional solution for any given d by iteratively applying our algorithm to the null space of the d l dimensional solution we also show how this result can be used to improve upon the outcomes provided by existing algorithms and derive a low computational cost linear approximation extensive experimental validations are provided to demonstrate the use of these algorithms in classification data analysis and visualization

face recognition using laplacianfaces we propose an appearance based face recognition method called the laplacianface approach by using locality preserving projections lpp the face images are mapped into a face subspace for analysis different from principal component analysis pca and linear discriminant analysis lda which effectively see only the euclidean structure of face space lpp finds an embedding that preserves local information and obtains a face subspace that best detects the essential face manifold structure the laplacianfaces are the optimal linear approximations to the eigenfunctions of the laplace beltrami operator on the face manifold in this way the unwanted variations resulting from changes in lighting facial expression and pose may be eliminated or reduced theoretical analysis shows that pca lda and lpp can be obtained from different graph models we compare the proposed laplacianface approach with eigenface and fisherface methods on three different face data sets experimental results suggest that the proposed laplacianface approach provides a better representation and achieves lower error rates in face recognition

finite elements for elliptic problems with stochastic coefficients abstract we describe a deterministic finite element fe solution algorithm for a stochastic elliptic boundary value problem sbvp whose coefficients are assumed to be random fields with finite second moments and known piecewise smooth two point spatial correlation function separation of random and deterministic variables parametrization of the uncertainty is achieved via a karhunen loeve kl expansion an o n log n algorithm for the computation of the kl eigenvalues is presented based on a kernel independent fast multipole method fmm truncation of the kl expansion gives an m 1 wiener polynomial chaos pc expansion of the stochastic coefficient and is shown to lead to a high dimensional deterministic boundary value problem dbvp analyticity of its solution in the stochastic variables with sharp bounds for the domain of analyticity are used to prescribe variable stochastic polynomial degree r r1 rm in an m r wiener pc expansion for the approximate solution pointwise error bounds for the fem approximations of kl eigenpairs the truncation of the kl expansion and the fe solution to the dbvp are given numerical examples show that m depends on the spatial correlation length of the random diffusion coefficient the variable polynomial degree r in pc stochastic galerkin fem allows to handle kl expansions with m up to 30 and r1 up to 10 in moderate time

taylor approximation and variance reduction for pde constrained optimal control under uncertainty abstract in this work we develop a scalable computational framework for the solution of pde constrained optimal control problems under high dimensional uncertainty specifically we consider a mean variance formulation of the control objective and employ a taylor expansion with respect to the uncertain parameter field either to directly approximate the control objective or as a control variate for variance reduction the expressions for the mean and variance of the taylor approximation are known analytically although their evaluation requires efficient computation of the trace of the preconditioned hessian of the control objective we propose to estimate this trace by solving a generalized eigenvalue problem using a randomized algorithm that only requires the action of the hessian on a small number of random directions then the computational work does not depend on the nominal dimension of the uncertain parameter but depends only on the effective dimension i e the rank of the preconditioned hessian thus ensuring scalability to high dimensional problems moreover to increase the estimation accuracy of the mean and variance of the control objective by the taylor approximation we use it as a control variate for variance reduction which results in considerable computational savings several orders of magnitude compared to a plain monte carlo method in summary our approach amounts to solving an optimal control problem constrained by the original pde the generalized eigenvalue equations at a small number of eigenfunctions and a set of linearized pdes that arise from the computation of the gradient and hessian of the control objective with respect to the uncertain parameter we use the lagrangian formalism to derive expressions for the gradient with respect to the control and apply a gradient based optimization method to solve the problem we demonstrate the accuracy efficiency and scalability of the proposed computational method for two examples with high dimensional uncertain parameters subsurface flow in a porous medium modeled as an elliptic pde and turbulent jet flow modeled by the reynolds averaged navier stokes equations coupled with a nonlinear advection diffusion equation characterizing model uncertainty in particular for the latter more challenging example we show scalability of our algorithm up to one million parameters resulting from discretization of the uncertain parameter field

an optimization criterion for generalized discriminant analysis on undersampled problems an optimization criterion is presented for discriminant analysis the criterion extends the optimization criteria of the classical linear discriminant analysis lda through the use of the pseudoinverse when the scatter matrices are singular it is applicable regardless of the relative sizes of the data dimension and sample size overcoming a limitation of classical lda the optimization problem can be solved analytically by applying the generalized singular value decomposition gsvd technique the pseudoinverse has been suggested and used for undersampled problems in the past where the data dimension exceeds the number of data points the criterion proposed in this paper provides a theoretical justification for this procedure an approximation algorithm for the gsvd based approach is also presented it reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster this reduced problem yields much smaller matrices to which the gsvd can be applied efficiently experiments on text data with up to 7 000 dimensions show that the approximation algorithm produces results that are close to those produced by the exact algorithm

k local hyperplane distance nearest neighbor classifier oriented local discriminant analysis k local hyperplane distance nearest neighbor hknn classifier is an improved k nearest neighbor knn algorithm that has been successfully applied to pattern classification this paper embeds the decision rule of hknn classifier into the discriminant analysis model to develop a new feature extractor the obtained feature extractor is called k local hyperplane distance nearest neighbor classifier oriented local discriminant analysis holda in which a regularization item is imposed on the original hknn algorithm to obtain a more reliable distance metric based on this distance metric the homo class and hetero class local scatters are characterized in holda by maximizing the ratio of the hetero class local scatter to the homo class local scatter we obtain a subspace which is suitable for feature extraction and classification in general this paper provides a framework for building a feature extractor from the decision rule of a classifier by this means the feature extractor and classifier can be seamlessly integrated experimental results on four databases demonstrate that the integrated pattern recognition system is effective

on the relation between discriminant analysis and mutual information for supervised linear feature extraction this paper provides a unifying view of three discriminant linear feature extraction methods linear discriminant analysis heteroscedastic discriminant analysis and maximization of mutual information we propose a model independent reformulation of the criteria related to these three methods that stresses their similarities and elucidates their differences based on assumptions for the probability distribution of the classification data we obtain sufficient conditions under which two or more of the above criteria coincide it is shown that these conditions also suffice for bayes optimality of the criteria our approach results in an information theoretic derivation of linear discriminant analysis and heteroscedastic discriminant analysis finally regarding linear discriminant analysis we discuss its relation to multidimensional independent component analysis and derive suboptimality bounds based on information theory

galerkin finite element approximations of stochastic elliptic partial differential equations we describe and analyze two numerical methods for a linear elliptic problem with stochastic coefficients and homogeneous dirichlet boundary conditions here the aim of the com putations is to approximate statistical moments of the solution and in particular we give a priori error estimates for the computation of the expected value of the solution the first method gener ates independent identically distributed approximations of the solution by sampling the coefficients of the equation and using a standard galerkin finite element variational formulation the monte carlo method then uses these approximations to compute corresponding sample averages the sec ond method is based on a finite dimensional approximation of the stochastic coefficients turning the original stochastic problem into a deterministic parametric elliptic problem a galerkin finite element method of either the h o rp version then approximates the corresponding deterministic solution yielding approximations of the desired statistics we present a priori error estimates and include a comparison of the computational work required by each numerical approximation to achieve a given accuracy this comparison suggests intuitive conditions for an optimal selection of the numerical approximation

on image analysis by moments research has been performed investigating the use of moments for pattern recognition in recent years the basic problem of the influence of discretization and noise on moment accuracy as object descriptors has been barely investigated in this paper the detailed error analysis involved in the moment method is discussed several new techniques to increase the accuracy and efficiency of moment descriptor are proposed we utilize these results for the problem of image reconstruction from the orthogonal legendre moments computed from discrete and noisy data the automatic selection of an optimal number of moments is also discussed

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

finite element approximations of stochastic optimal control problems constrained by stochastic elliptic pdes abstract in this paper we study mathematically and computationally optimal control problems for stochastic elliptic partial differential equations the control objective is to minimize the expectation of a tracking cost functional and the control is of the deterministic distributed type the main analytical tool is the wiener ito chaos or the karhunen loeve expansion mathematically we prove the existence of an optimal solution we establish the validity of the lagrange multiplier rule and obtain a stochastic optimality system of equations we represent the input data in their wiener ito chaos expansions and deduce the deterministic optimality system of equations computationally we approximate the optimality system through the discretizations of the probability space and the spatial space by the finite element method we also derive error estimates in terms of both types of discretizations

stochastic optimal robin boundary control problems of advection dominated elliptic equations in this work we deal with a stochastic optimal robin boundary control problem constrained by an advection diffusion reaction elliptic equation with advection dominated term we assume that the uncertainty comes from the advection field and consider a stochastic robin boundary condition as control function a stochastic saddle point system is formulated and proved to be equivalent to the first order optimality system for the optimal control problem based on which we provide the existence and uniqueness of the optimal solution as well as some results on stochastic regularity with respect to the random variables stabilized finite element approximations in physical space and collocation approximations in stochastic space are applied to discretize the optimality system a global error estimate in the product of physical space and stochastic space for the numerical approximation is derived illustrative numerical experiments are provided

generalized discriminant analysis using a kernel approach we present a new method that we call generalized discriminant analysis gda to deal with nonlinear discriminant analysis using kernel function operator the underlying theory is close to the support vector machines svm insofar as the gda method provides a mapping of the input vectors into high dimensional feature space in the transformed space linear properties make it easy to extend and generalize the classical linear discriminant analysis lda to nonlinear discriminant analysis the formulation is expressed as an eigenvalue problem resolution using a different kernel one can cover a wide class of nonlinearities for both simulated data and alternate kernels we give classification results as well as the shape of the decision function the results are confirmed using real data to perform seed classification

multilevel and weighted reduced basis method for stochastic optimal control problems constrained by stokes equations in this paper we develop and analyze a multilevel weighted reduced basis method for solving stochastic optimal control problems constrained by stokes equations we prove the analytic regularity of the optimal solution in the probability space under certain assumptions on the random input data the finite element method and the stochastic collocation method are employed for the numerical approximation of the problem in the deterministic space and the probability space respectively resulting in many large scale optimality systems to solve in order to reduce the unaffordable computational effort we propose a reduced basis method using a multilevel greedy algorithm in combination with isotropic and anisotropic sparse grid techniques a weighted a posteriori error bound highlights the contribution stemming from each method numerical tests on stochastic dimensions ranging from 10 to 100 demonstrate that our method is very efficient especially for solving high dimensional and large scale optimization problems

discriminant waveletfaces and nearest feature classifiers for face recognition feature extraction discriminant analysis and classification rules are three crucial issues for face recognition we present hybrid approaches to handle three issues together for feature extraction we apply the multiresolution wavelet transform to extract the waveletface we also perform the linear discriminant analysis on waveletfaces to reinforce discriminant power during classification the nearest feature plane nfp and nearest feature space nfs classifiers are explored for robust decisions in presence of wide facial variations their relationships to conventional nearest neighbor and nearest feature line classifiers are demonstrated in the experiments the discriminant waveletface incorporated with the nfs classifier achieves the best face recognition performance

semi supervised linear discriminant analysis for dimension reduction and classification when facing high dimensional data dimension reduction is necessary before classification among dimension reduction methods linear discriminant analysis lda is a popular one that has been widely used lda aims to maximize the ratio of the between class scatter and total data scatter in projected space and the label of each data is necessary however in real applications the labeled data are scarce and unlabeled data are in large quantity so lda is hard to be used under such case in this paper we propose a novel method named semi supervised linear discriminant analysis slda which can use limited number of labeled data and a quantity of the unlabeled ones for training so that lda can accommodate to the situation of a few labeled data available assuming that f represents the calculated class indicator matrix of the training data and y denotes the true label of the labeled data the objective function contains two parts one is the criterion of lda which is a function of projection w and a class indicator matrix f the other is the difference between the true data label and calculated label of these labeled data as far as we know there is no closed form solution to the objective function to solve such problem we develop an iterative algorithm which calculates the class indicator matrix and the projection alternatively the convergence of the proposed iterative algorithm is proved and confirmed by experiments the experimental results on eight datasets show that the performance of slda is superior to that of traditional lda and some state of the art semi supervised algorithms highlightsa semi supervised variant of lda named semi supervised lda is proposed our method can use limited number of labeled data and a quantity of the unlabeled ones for training we develop an iterative algorithm which calculates the class indicator matrix and the projection alternatively

finite dimensional approximation of a class of constrained nonlinear optimal control problems an abstract framework for the analysis and approximation of a class of nonlinear optimal control and optimization problems is constructed nonlinearities occur in both the objective functional and the constraints the framework includes an abstract nonlinear optimization problem posed on infinite dimensional spaces an approximate problem posed on finite dimensional spaces together with a number of hypotheses concerning the two problems the framework is used to show that optimal solutions exist to show that lagrange multipliers may be used to enforce the constraints to derive an optimality system from which optimal states and controls may be deduced and to derive existence results and error estimates for solutions of the approximate problem the abstract framework and the results derived from that framework are then applied to three concrete control or optimization problems and their approximation by finite element methods the first involves the von k arm an plate equations of nonlinear elasticity the second the ginzburg landau equations of superconductivity and the third the navier stokes equations for incompressible viscous flows