infinite dimensional integration and the multivariate decomposition method abstract we further develop the multivariate decomposition method mdm for the lebesgue integration of functions of infinitely many variables x 1 x 2 x 3 with respect to a corresponding product of a one dimensional probability measure the method is designed for functions that admit a dominantly convergent decomposition f u f u where u runs over all finite subsets of positive integers and for each u i 1 i k the function f u depends only on x i 1 x i k although a number of concepts of infinite dimensional integrals have been used in the literature questions of uniqueness and compatibility have mostly not been studied we show that under appropriate convergence conditions the lebesgue integral equals the anchored integral independently of the anchor for approximating the integral the mdm assumes that point values of f u are available for important subsets u at some known cost in this paper we introduce a new setting in which it is assumed that each f u belongs to a normed space f u and that bounds b u on f u f u are known this contrasts with the assumption in many papers that weights u appearing in the norm of the infinite dimensional function space are somehow known often such weights u were determined by minimizing an error bound depending on the b u the u and the chosen algorithm resulting in weights that depend on the algorithm in contrast in this paper only the bounds b u are assumed to be known we give two examples in which we specialize the mdm in the first case f u is the u fold tensor product of an anchored reproducing kernel hilbert space in the second case it is a particular non hilbert space for integration over an unbounded domain

sparse ensembles using weighted combination methods based on linear programming an ensemble of multiple classifiers is widely considered to be an effective technique for improving accuracy and stability of a single classifier this paper proposes a framework of sparse ensembles and deals with new linear weighted combination methods for sparse ensembles sparse ensemble is to sparsely combine the outputs of multiple classifiers by using a sparse weight vector when the continuous outputs of multiple classifiers are provided in our methods the problem of solving sparse weight vector can be formulated as linear programming problems in which the hinge loss or and the 1 norm regularization are exploited both the hinge loss and the 1 norm regularization are techniques inducing sparsity used in machine learning we only ensemble classifiers with nonzero weight coefficients in these lp based methods the ensemble training error is minimized while the weight vector of ensemble learning is controlled which can be thought as implementing the structure risk minimization rule and naturally explains good performance of these methods the promising experimental results over uci data sets and the radar high resolution range profile data are presented

ensemble pruning via semi definite programming an ensemble is a group of learning models that jointly solve a problem however the ensembles generated by existing techniques are sometimes unnecessarily large which can lead to extra memory usage computational costs and occasional decreases in effectiveness the purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as or better than the original ensemble this subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive various heuristic methods have been developed to obtain an approximate solution however most of the existing heuristics use simple greedy search as the optimization method which lacks either theoretical or empirical quality guarantees in this paper the ensemble subset selection problem is formulated as a quadratic integer programming problem by applying semi definite programming sdp as a solution technique we are able to get better approximate solutions computational experiments show that this sdp based pruning algorithm outperforms other heuristics in the literature its application in a classifier sharing study also demonstrates the effectiveness of the method

decision combination in multiple classifier systems a multiple classifier system is a powerful solution to difficult pattern recognition problems involving large class sets and noisy input because it allows simultaneous use of arbitrary feature descriptors and classification procedures decisions by the classifiers can be represented as rankings of classifiers and different instances of a problem the rankings can be combined by methods that either reduce or rerank a given set of classes an intersection method and union method are proposed for class set reduction three methods based on the highest rank the borda count and logistic regression are proposed for class set reranking these methods have been tested in applications of degraded machine printed characters and works from large lexicons resulting in substantial improvement in overall correctness

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

a data driven stochastic collocation approach for uncertainty quantification in mems this work presents a data driven stochastic collocation approach to include the effect of uncertain design parameters during complex multi physics simulation of micro electromechanical systems mems the proposed framework comprises of two key steps first probabilistic characterization of the input uncertain parameters based on available experimental information and second propagation of these uncertainties through the predictive model to relevant quantities of interest the uncertain input parameters are modeled as independent random variables for which the distributions are estimated based on available experimental observations using a nonparametric diffusion mixing based estimator botev nonparametric density estimation via diffusion mixing technical report 2007 the diffusion based estimator derives from the analogy between the kernel density estimation kde procedure and the heat dissipation equation and constructs density estimates that are smooth and asymptotically consistent the diffusion model allows for the incorporation of the prior density and leads to an improved density estimate in comparison with the standard kde approach as demonstrated through several numerical examples following the characterization step the uncertainties are propagated to the output variables using the stochastic collocation approach based on sparse grid interpolation smolyak soviet math dokl 1963 4 240 243 the developed framework is used to study the effect of variations in young s modulus induced as a result of variations in manufacturing process parameters or heterogeneous measurements on the performance of a mems switch copyright 2010 john wiley sons ltd

a multiscale preconditioner for stochastic mortar mixed finite elements the aim of this paper is to introduce a new approach to efficiently solve sequences of problems that typically arise when modeling flow in stochastic porous media the governing equations are based on darcy s law with a stochastic permeability field starting from a specified covariance relationship the log permeability is decomposed using a truncated karhunen loeve expansion multiscale mortar mixed finite element approximations are used in the spatial domain and a nonintrusive sampling method is used in the stochastic dimensions a multiscale mortar flux basis is computed for a single permeability called a training permeability that captures the main characteristics of the porous media and is used as a preconditioner for each stochastic realization we prove that the condition number of the preconditioned interface operator is independent of the subdomain mesh size and the mortar mesh size computational results confirm that our approach provides an efficient means to quantify the uncertainty for stochastic flow in porous media

minimal multi element stochastic collocation for uncertainty quantification of discontinuous functions we propose a multi element stochastic collocation method that can be applied in high dimensional parameter space for functions with discontinuities lying along manifolds of general geometries the key feature of the method is that the parameter space is decomposed into multiple elements defined by the discontinuities and thus only the minimal number of elements are utilized on each of the resulting elements the function is smooth and can be approximated using high order methods with fast convergence properties the decomposition strategy is in direct contrast to the traditional multi element approaches which define the sub domains by repeated splitting of the axes in the parameter space such methods are more prone to the curse of dimensionality because of the fast growth of the number of elements caused by the axis based splitting the present method is a two step approach firstly a discontinuity detector is used to partition parameter space into disjoint elements in each of which the function is smooth the detector uses an efficient combination of the high order polynomial annihilation technique along with adaptive sparse grids and this allows resolution of general discontinuities with a smaller number of points when the discontinuity manifold is low dimensional after partitioning an adaptive technique based on the least orthogonal interpolant is used to construct a generalized polynomial chaos surrogate on each element the adaptive technique reuses all information from the partitioning and is variance suppressing we present numerous numerical examples that illustrate the accuracy efficiency and generality of the method when compared against standard locally adaptive sparse grid methods the present method uses many fewer number of collocation samples and is more accurate

quantum mechanics of charged particles in random electromagnetic fields in this paper we consider the 0 asymptotics for the solutions to stochastic schrodinger equations for quantum mechanical particles in random electromagnetic fields in rn we obtain semi classical expansions for their solutions up to any order in l2 rn a s by using a stochastic hamilton jacobi equation and a stochastic continuity equation we conclude that as 0 the stochastic quantum mechanics with random electromagnetic fields tends to stochastic classical mechanics

cross validation in principal component analysis summary this paper describes a form of cross validation in the context of principal component analysis which has a number of useful aspects as regards multivariate data inspection and description topics covered include choice of dimensionality identification of influential observations and selection of important variables the methods are motivated by and illustrated on a well known data set 1 data set and objectives jeffers 1967 described two detailed multivariate case studies one of which concerned 19 variables measured on each of 40 winged aphids alatee adelges that had been caught in a light trap the 19 variables are listed in table 1 principal component analysis pca was used to examine the structure in the data and if possible to answer the following questions i how many dimensions of the individuals are being measured ii how many distinct taxa are present in the habitat iii which variables among the 19 are redundant for distinguishing between taxa and which must be retained in future work of the 19 variables 14 are length or width measurements four are counts and one anal fold is a presence absence variable scored 0 or 1 in view of this disparity in variable type jeffers elected to standardise the data and thus effect the pca by finding the latent roots and vectors of the correlation rather than covariance matrix of the data the elements of each latent vector provide the coefficients of one of 19 linear combinations of the standardised original variables that successively maximise sample variance subject to being orthogonal with each other and the corresponding latent root is the sample variance of that linear combination the 19 observations for each aphid were subjected to each of these 19 linear transformations to form the 19 principal component scores for that aphid the above questions were then answered as follows i the latent roots of the correlation matrix were as given in table 1 the four largest comprise 73 0 12 5 3 9 and 2 6 respectively of the total variance 19 0 of the standardised variables the dimensionality of the data was therefore taken to be 2 ii when the scores of the first two principal components for the 40 aphids were plotted against orthogonal axes the resulting 40 points divided into four groups as shown in figure 1 hence four distinct species were identified for the aphids iii from consideration of the size of coefficients in the first three principal components it was concluded that only the four variables length of tibia number of ovipositor

multi fidelity non intrusive polynomial chaos based on regression abstract in this paper we present a multi fidelity mf extension of non intrusive polynomial chaos based on regression point collocation for uncertainty quantification purposes the proposed method uses the principle of a global correction function from a previous similar method that uses spectral projection to estimate the coefficients due to its usage of regression to estimate the coefficients the present method offers high flexibility in the sampling and generation of the polynomial basis the method takes advantage of a nested sampling plan to create the samples for the low fidelity lf and correction expansions where the high fidelity hf samples are a subset of the lf ones to build the polynomial basis a total order or hyperbolic truncation strategy is used with a highly flexible combination of the lf and correction polynomial expansions the method is demonstrated on some artificial test problems and aerodynamic problems of the euler flow around an airfoil and common three dimensional research models in order to derive the strategies for successful mf approximation the effect of the correlation and the errors between the lf and hf functions is also studied the results show that high correlation and moderately low errors are important to improve the mf approximation s accuracy on a common research model problem the mf approach with partially converged simulations as the lf samples can successfully reduce the computational cost to about 40 for similar accuracy compared to an approach using a single hf expansion

realization of compact lie algebras in kahler manifolds the berezin quantization on a simply connected homogeneous kahler manifold which is considered as a phase space for a dynamical system enables a description of the quantal system in a finite dimensional hilbert space of holomorphic functions corresponding to generalized coherent states the lie algebra associated with the manifold symmetry group is given in terms of first order differential operators in the classical theory the lie algebra is represented by the momentum maps which are functions on the manifold and the lie product is the poisson bracket given by the kahler structure the kahler potentials are constructed for the manifolds related to all compact semi simple lie groups the complex coordinates are introduced by means of the borel method the kahler structure is obtained explicitly for any unitary group representation the cocycle functions for the lie algebra and the killing vector fields on the manifold are also obtained

application of random walk methods to wave propagation we present an approach to problems of diffraction that has its roots in a number of well established theories such as the geometric theory of diffraction the methods of feynman s path integrals and wiener s functional integrals and the theory of stochastic processes we start our analysis of the helmholtz equation by closely following the scheme of the ray method but instead of approximating the resulting second order auxiliary equation by a first order transport equation we study the original auxiliary equation and obtain its exact solution as the mathematical expectation of a specified functional in the space of brownian trajectories with the wiener probabilistic measure the obtained solution appears to be a direct improvement of the ray method approximation to the exact solution of the helmholtz equation and it is shown to admit efficient numerical evaluation work supported by nsf grant 9730779

learning similarity with cosine similarity ensemble this paper proposes a cosine similarity ensemble cse method to learn similarity cse is a selective ensemble and combines multiple cosine similarity learners a learner redefines the pattern vectors and determines its threshold adaptively experimental results show the superiority of cse there is no doubt that similarity is a fundamental notion in the field of machine learning and pattern recognition how to represent and measure similarity appropriately is a pursuit of many researchers many tasks such as classification and clustering can be accomplished perfectly when a similarity metric is well defined cosine similarity is a widely used metric that is both simple and effective this paper proposes a cosine similarity ensemble cse method for learning similarity in cse diversity is guaranteed by using multiple cosine similarity learners each of which makes use of a different initial point to define the pattern vectors used in its similarity measures the cse method is not limited to measuring similarity using only pattern vectors that start at the origin in addition the thresholds of these separate cosine similarity learners are adaptively determined the idea of using a selective ensemble is also implemented in cse and the proposed cse method outperforms other compared methods on various data sets

locally adaptive metric nearest neighbor classification nearest neighbor classification assumes locally constant class conditional probabilities this assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality severe bias can be introduced under these conditions when using the nearest neighbor rule we propose a locally adaptive nearest neighbor classification method to try to minimize bias we use a chi squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones as a result the class conditional probabilities are smoother in the modified neighborhoods whereby better classification performance can be achieved the efficacy of our method is validated and compared against other techniques using both simulated and real world data

optimizing the data dependent kernel under a unified kernel optimization framework the kernel functions play a central role in kernel methods accordingly over the years the optimization of kernel functions has been a promising research area ideally fisher discriminant criteria can be used as an objective function to optimize the kernel function to augment the margin between different classes unfortunately fisher criteria are optimal only in the case that all the classes are generated from underlying multivariate normal distributions of common covariance matrix but different means and each class is expressed by a single cluster due to the assumptions fisher criteria obviously are not a suitable choice as a kernel optimization rule in some applications such as the multimodally distributed data in order to solve this problem recently many improved discriminant criteria dc have been also developed therefore to apply these discriminant criteria to kernel optimization in this paper based on a data dependent kernel function we propose a unified kernel optimization framework which can use any discriminant criteria formulated in a pairwise manner as the objective functions under the kernel optimization framework to employ different discriminant criteria one has to only change the corresponding affinity matrices without having to resort to any complex derivations in feature space experimental results based on some benchmark data demonstrate the efficiency of our method

a theoretical study on six classifier fusion strategies we look at a single point in feature space two classes and l classifiers estimating the posterior probability for class spl omega sub 1 assuming that the estimates are independent and identically distributed normal or uniform we give formulas for the classification error for the following fusion methods average minimum maximum median majority vote and oracle

a novel combining classifier method based on variational inference in this paper we propose a combining classifier method based on the bayesian inference framework specifically the outputs of base classifiers called level1 data or meta data are utilized in a combiner to produce the final classification in our ensemble system each class in the training set induces a distribution on the level1 data which is modeled by a multivariate gaussian distribution traditionally the parameters of the gaussian are estimated using a maximum likelihood approach however maximum likelihood estimation cannot be applied since the covariance matrix of level1 data of each class is not full rank instead we propose to estimate the multivariate gaussian distribution of level1 data of each class by using the variational inference method experiments conducted on eighteen uci machine learning repository datasets and a selected 10 class clef2009 medical imaging database demonstrated the advantage of our method compared with several well known ensemble methods we proposed a variational inference method for multivariate gaussian distribution estimation we proposed a novel combining classifier method by employing variational inference on base classifiers outputs the proposed method is highly competitive to many state of the arts ensemble classifier methods

path integral solution of the schrodinger equation in curvilinear coordinates a straightforward procedure a new axiomatic formulation of path integrals is used to construct a path integral solution of the schrodinger equation in curvilinear coordinates an important feature of the formalism is that a coordinate transformation in the variables of the wavefunction does not imply a change of variable of integration in the path integral consequently a transformation from euclidean to curvilinear coordinates is simple to handle there is no need to introduce quantum corrections into the action functional furthermore the paths are differentiable hence issues related to stochastic paths do not arise the procedure for constructing the path integral solution of the schrodinger equation is straightforward the case of the schrodinger equation in spherical coordinates for a free particle is presented in detail

an equivalence between sparse approximation and support vector machines in the first part of this paper we show a similarity between the principle of structural risk minimization principle srm vapnik 1982 and the idea of sparse approximation as defined in chen donoho and saunders 1995 and olshausen and field 1996 then we focus on two specific approximate implementations of srm and sparse approximation which have been used to solve the problem of function approximation for srm we consider the support vector machine technique proposed by v vapnik and his team at at t bell labs and for sparse approximation we consider a modification of the basis pursuit de noising algorithm proposed by chen donoho and saunders 1995 we show that under certain conditions these two techniques are equivalent they give the same solution and they require the solution of the same quadratic programming problem

selective ensemble of svdds with renyi entropy based diversity measure abstract in this paper a novel selective ensemble strategy for support vector data description svdd using the renyi entropy based diversity measure is proposed to deal with the problem of one class classification in order to obtain compact classification boundary the radius of ensemble is defined as the inner product of the vector of combination weights and the vector of the radii of svdds to make the center of ensemble achieve the optimal position the renyi entropy of the kernelized distances between the images of samples and the center of ensemble in the high dimensional feature space is defined as the diversity measure moreover to fulfill the selective ensemble an l 1 norm based regularization term is introduced into the objective function of the proposed ensemble the optimal combination weights can be iteratively obtained by the half quadratic optimization technique experimental results on two synthetic data sets and twenty benchmark data sets demonstrate that the proposed selective ensemble method is superior to the single svdd and the other four related ensemble approaches

exact constructions of square root helmholtz operator symbols the focusing quadratic profile operator symbols play a pivotal role in both the exact well posed one way reformulation of solving the elliptic helmholtz equation and the construction of the generalized bremmer coupling series the inverse square root and square root helmholtz operator symbols are the initial quantities of interest in both formulations in addition to providing the theoretical framework for the development and implementation of the parabolic equation pe method in wave propagation modeling exact standard left and weyl symbol constructions are presented for both the inverse square root and square root helmholtz operators in the case of the focusing quadratic profile in one transverse spatial dimension extending and ultimately unifying the previously published corresponding results for the defocusing quadratic case j math phys 33 1887 1914 1992 both i spectral modal summation representations and ii contour integral representations exploiting the underlying periodicity of the associated quantum mechanical harmonic oscillator problem are derived and ultimately related through the propagating and nonpropagating contributions to the operator symbol high and low frequency asymptotic operator symbol expansions are given along with the exact symbol representations for the corresponding operator rational approximations which provide the basis for the practical computational realization of the pe method moreover while the focusing quadratic profile is in some respects nonphysical the corresponding helmholtz operator symbols nevertheless establish canonical symbol features for more general profiles containing locally quadratic wells

probabilistic collocation for period 1 limit cycle oscillations in this paper probabilistic collocation for limit cycle oscillations pclco is proposed probabilistic collocation pc is a non intrusive approach to compute the polynomial chaos description of uncertainty numerically polynomial chaos can require impractical high orders to approximate long term time integration problems due to the fast increase of required polynomial chaos order with time pclco is a pc formulation for modeling the long term stochastic behavior of dynamical systems exhibiting a periodic response i e a limit cycle oscillation lco in the pc method deterministic time series are computed at collocation points in probability space in pclco pc is applied to a time independent parametrization of the periodic response of the deterministic solves instead of to the time dependent functions themselves due to the time independent parametrization the accuracy of pclco is independent of time the approach is applied to period 1 oscillations with one main frequency subject to a random parameter numerical results are presented for the harmonic oscillator a two dof airfoil flutter model and the fluid structure interaction of an elastically mounted cylinder

density induced margin support vector machines this paper proposes a new classifier called density induced margin support vector machines dmsvms dmsvms belong to a family of svm like classifiers thus dmsvms inherit good properties from support vector machines svms e g unique and global solution and sparse representation for the decision function for a given data set dmsvms require to extract relative density degrees for all training data points these density degrees can be taken as relative margins of corresponding training data points moreover we propose a method for estimating relative density degrees by using the k nearest neighbor method we also show the upper bound on the leave out one error of dmsvms for a binary classification problem and prove it promising results are obtained on toy as well as real world data sets

optimal control with stochastic pde constraints and uncertain controls abstract the optimal control of problems that are constrained by partial differential equations with uncertainties and with uncertain controls is addressed the lagrangian that defines the problem is postulated in terms of stochastic functions with the control function possibly decomposed into an unknown deterministic component and a known zero mean stochastic component the extra freedom provided by the stochastic dimension in defining cost functionals is explored demonstrating the scope for controlling statistical aspects of the system response one shot stochastic finite element methods are used to find approximate solutions to control problems it is shown that applying the stochastic collocation finite element method to the formulated problem leads to a coupling between stochastic collocation points when a deterministic optimal control is considered or when moments are included in the cost functional thereby forgoing the primary advantage of the collocation method over the stochastic galerkin method for the considered problem the application of the presented methods is demonstrated through a number of numerical examples the presented framework is sufficiently general to also consider a class of inverse problems and numerical examples of this type are also presented

parameter and state model reduction for large scale statistical inverse problems a greedy algorithm for the construction of a reduced model with reduction in both parameter and state is developed for an efficient solution of statistical inverse problems governed by partial differential equations with distributed parameters large scale models are too costly to evaluate repeatedly as is required in the statistical setting furthermore these models often have high dimensional parametric input spaces which compounds the difficulty of effectively exploring the uncertainty space we simultaneously address both challenges by constructing a projection based reduced model that accepts low dimensional parameter inputs and whose model evaluations are inexpensive the associated parameter and state bases are obtained through a greedy procedure that targets the governing equations model outputs and prior information the methodology and results are presented for groundwater inverse problems in one and two dimensions

on stability and monotonicity requirements of finite difference approximations of stochastic conservation laws with random viscosity on stability and monotonicity requirements of finite difference approximations of stochastic conservation laws with random viscosity

certification of projection based reduced order modelling in computational homogenisation by the constitutive relation error in this paper we propose upper and lower error bounding techniques for reduced order modelling applied to the computational homogenisation of random composites the upper bound relies on the construction of a reduced model for the stress field upon ensuring that the reduced stress satisfies the equilibrium in the finite element sense the desired bounding property is obtained the lower bound is obtained by defining a hierarchical enriched reduced model for the displacement we show that the sharpness of both error estimates can be seamlessly controlled by adapting the parameters of the corresponding reduced order model

texture analysis using graphs generated by deterministic partially self avoiding walks texture is one of the most important visual attributes for image analysis it has been widely used in image analysis and pattern recognition a partially self avoiding deterministic walk has recently been proposed as an approach for texture analysis with promising results this approach uses walkers called tourists to exploit the gray scale image contexts in several levels here we present an approach to generate graphs out of the trajectories produced by the tourist walks the generated graphs embody important characteristics related to tourist transitivity in the image computed from these graphs the statistical position degree mean and dispersion entropy of two vertices with the same degree measures are used as texture descriptors a comparison with traditional texture analysis methods is performed to illustrate the high performance of this novel approach

risk averse structural topology optimization under random fields using stochastic expansion methods abstract this work proposes a level set based approach for solving risk averse structural topology optimization problems considering random field loading and material uncertainty the use of random fields increases the dimensionality of the stochastic domain which poses several computational challenges related to the minimization of the excess probability as a measure of risk awareness this problem is addressed both from the theoretical and numerical viewpoints first an existence result under a typical geometrical constraint on the set of admissible shapes is proved second a level set continuous approach to find the numerical solution of the problem is proposed since the considered cost functional has a discontinuous integrand the numerical approximation of the functional and its sensitivity combine an adaptive anisotropic polynomial chaos pc approach with a monte carlo mc sampling method for uncertainty propagation furthermore to address the increment of dimensionality induced by the random field an anisotropic sparse grid stochastic collocation method is used for the efficient computation of the pc coefficients a key point is that the non intrusive nature of such an approach facilitates the use of high performance computing hpc to alleviate the computational burden of the problem several numerical experiments including random field loading and material uncertainty are presented to show the feasibility of the proposal

exploration of classification confidence in ensemble learning ensemble learning has attracted considerable attention owing to its good generalization performance the main issues in constructing a powerful ensemble include training a set of diverse and accurate base classifiers and effectively combining them ensemble margin computed as the difference of the vote numbers received by the correct class and the another class received with the most votes is widely used to explain the success of ensemble learning this definition of the ensemble margin does not consider the classification confidence of base classifiers in this work we explore the influence of the classification confidence of the base classifiers in ensemble learning and obtain some interesting conclusions first we extend the definition of ensemble margin based on the classification confidence of the base classifiers then an optimization objective is designed to compute the weights of the base classifiers by minimizing the margin induced classification loss several strategies are tried to utilize the classification confidences and the weights it is observed that weighted voting based on classification confidence is better than simple voting if all the base classifiers are used in addition ensemble pruning can further improve the performance of a weighted voting ensemble we also compare the proposed fusion technique with some classical algorithms the experimental results also show the effectiveness of weighted voting with classification confidence

infinite dimensional integration on weighted hilbert spaces we study the numerical integration problem for functions with infinitely many variables the functions we want to integrate are from a reproducing kernel hilbert space which is endowed with a weighted norm we study the worst case e complexity which is defined as the minimal cost among all algorithms whose worst case error over the hilbert space unit ball is at most e here we assume that the cost of evaluating a function depends polynomially on the number of active variables the infinite dimensional integration problem is polynomially tractable if the e complexity is bounded by a constant times a power of 1 e the smallest such power is called the exponent of tractability first we study finite order weights we provide improved lower bounds for the exponent of tractability for general finite order weights and improved upper bounds for three newly defined classes of finite order weights the constructive upper bounds are obtained by multilevel algorithms that use for each level quasimonte carlo integration points whose projections onto specific sets of coordinates exhibit a small discrepancy the newly defined finite intersection weights model the situation where each group of variables interacts with at most other groups of variables where is some fixed number for these weights we obtain a sharp upper bound this is the first class of weights for which the exact exponent of tractability is known for any possible decay of the weights and for any polynomial degree of the cost function for the other two classes of finite order weights our upper bounds are sharp if e g the decay of the weights is fast or slow enough we extend our analysis to the case of arbitrary weights in particular from our results for finite order weights we conclude a lower bound on the exponent of tractability for arbitrary weights and a constructive upper bound for product weights

time displaced interactions classical dynamics and path integral quantization the path integral was created to quantize systems whose dynamics are nonlocal in time and for which a hamiltonian formulation could not be found in this article we quantize a linear oscillator with an interaction that depends on the oscillator s position at past and future times the central issue is the classical boundary value problem for such an oscillator because two time functional boundary conditions are used to implement the quantization we take up related questions such as causality in this context and the general issue of stability for forward and inward data specifications

a robust bi orthogonal dynamically orthogonal method using the covariance pseudo inverse with application to stochastic flow problems abstract we develop a new robust methodology for the stochastic navier stokes equations based on the dynamically orthogonal do and bi orthogonal bo methods 1 2 3 both approaches are variants of a generalized karhunen loeve kl expansion in which both the stochastic coefficients and the spatial basis evolve according to system dynamics hence capturing the low dimensional structure of the solution the do and bo formulations are mathematically equivalent 3 but they exhibit computationally complimentary properties specifically the bo formulation may fail due to crossing of the eigenvalues of the covariance matrix while both bo and do become unstable when there is a high condition number of the covariance matrix or zero eigenvalues to this end we combine the two methods into a robust hybrid framework and in addition we employ a pseudo inverse technique to invert the covariance matrix the robustness of the proposed method stems from addressing the following issues in the do bo formulation i eigenvalue crossing we resolve the issue of eigenvalue crossing in the bo formulation by switching to the do near eigenvalue crossing using the equivalence theorem and switching back to bo when the distance between eigenvalues is larger than a threshold value ii ill conditioned covariance matrix we utilize a pseudo inverse strategy to invert the covariance matrix iii adaptivity we utilize an adaptive strategy to add remove modes to resolve the covariance matrix up to a threshold value in particular we introduce a soft threshold criterion to allow the system to adapt to the newly added removed mode and therefore avoid repetitive and unnecessary mode addition removal when the total variance approaches zero we show that the do bo formulation becomes equivalent to the evolution equation of the optimally time dependent modes 4 we demonstrate the capability of the proposed methodology with several numerical examples namely i stochastic burgers equation we analyze the performance of the method in the presence of eigenvalue crossing and zero eigenvalues ii stochastic kovasznay flow we examine the method in the presence of a singular covariance matrix and iii we examine the adaptivity of the method for an incompressible flow over a cylinder where for large stochastic forcing thirteen do bo modes are active

stochastic mechanics and the feynman integral the feynman integral is given a stochastic interpretation in the framework of nelson s stochastic mechanics employing a time symmetric variant of nelson s kinematics recently developed by the author

on tractability of path integration many applications require approximate values of path integrals a typical approach is to approximate the path integral by a high dimensional integral and apply a monte carlo randomized algorithm however monte carlo algorithm requires roughly e 2 integrand evaluations to provide an e approximation moreover the error bound of e is guaranteed only in a stochastic sense do we really need to use randomized algorithms for path integrals perhaps we can find a deterministic algorithm that is more effective even in the worst case setting to answer this question we study the worst case complexity of path integration which roughly speaking is defined as the minimal number of the integrand evaluations needed to compute an approximation with error at most e we consider path integration with respect to a gaussian measure and for various classes of integrands tractability of path integration means that the complexity depends polynomially on 1 e we show that for the class of r times frechet differentiab

an analysis of ensemble pruning techniques based on ordered aggregation several pruning strategies that can be used to reduce the size and increase the accuracy of bagging ensembles are analyzed these heuristics select subsets of complementary classifiers that when combined can perform better than the whole ensemble the pruning methods investigated are based on modifying the order of aggregation of classifiers in the ensemble in the original bagging algorithm the order of aggregation is left unspecified when this order is random the generalization error typically decreases as the number of classifiers in the ensemble increases if an appropriate ordering for the aggregation process is devised the generalization error reaches a minimum at intermediate numbers of classifiers this minimum lies below the asymptotic error of bagging pruned ensembles are obtained by retaining a fraction of the classifiers in the ordered ensemble the performance of these pruned ensembles is evaluated in several benchmark classification tasks under different training conditions the results of this empirical investigation show that ordered aggregation can be used for the efficient generation of pruned ensembles that are competitive in terms of performance and robustness of classification with computationally more costly methods that directly select optimal or near optimal subensembles

exact 1 norm support vector machines via unconstrained convex differentiable minimization support vector machines utilizing the 1 norm typically set up as linear programs mangasarian 2000 bradley and mangasarian 1998 are formulated here as a completely unconstrained minimization of a convex differentiable piecewise quadratic objective function in the dual space the objective function which has a lipschitz continuous gradient and contains only one additional finite parameter can be minimized by a generalized newton method and leads to an exact solution of the support vector machine problem the approach here is based on a formulation of a very general linear program as an unconstrained minimization problem and its application to support vector machine classification problems the present approach which generalizes both mangasarian 2004 and fung and mangasarian 2004 is also applied to nonlinear approximation where a minimal number of nonlinear kernel functions are utilized to approximate a function from a given number of function values

a stochastic mixed finite element heterogeneous multiscale method for flow in porous media a computational methodology is developed to efficiently perform uncertainty quantification for fluid transport in porous media in the presence of both stochastic permeability and multiple scales in order to capture the small scale heterogeneity a new mixed multiscale finite element method is developed within the framework of the heterogeneous multiscale method hmm in the spatial domain this new method ensures both local and global mass conservation starting from a specified covariance function the stochastic log permeability is discretized in the stochastic space using a truncated karhunen loeve expansion with several random variables due to the small correlation length of the covariance function this often results in a high stochastic dimensionality therefore a newly developed adaptive high dimensional stochastic model representation technique hdmr is used in the stochastic space this results in a set of low stochastic dimensional subproblems which are efficiently solved using the adaptive sparse grid collocation method asgc numerical examples are presented for both deterministic and stochastic permeability to show the accuracy and efficiency of the developed stochastic multiscale method

a sparse grid stochastic collocation method for partial differential equations with random input data this work proposes and analyzes a smolyak type sparse grid stochastic collocation method for the approximation of statistical quantities related to the solution of partial differential equations with random coefficients and forcing terms input data of the model to compute solution statistics the sparse grid stochastic collocation method uses approximate solutions produced here by finite elements corresponding to a deterministic set of points in the random input space this naturally requires solving uncoupled deterministic problems as in the monte carlo method if the number of random variables needed to describe the input data is moderately large full tensor product spaces are computationally expensive to use due to the curse of dimensionality in this case the sparse grid approach is still expected to be competitive with the classical monte carlo method therefore it is of major practical relevance to understand in which situations the sparse grid stochastic collocation method is more efficient than monte carlo this work provides error estimates for the fully discrete solution using l q norms and analyzes the computational efficiency of the proposed method in particular it demonstrates algebraic convergence with respect to the total number of collocation points and quantifies the effect of the dimension of the problem number of input random variables in the final estimates the derived estimates are then used to compare the method with monte carlo indicating for which problems the former is more efficient than the latter computational evidence complements the present theory and shows the effectiveness of the sparse grid stochastic collocation method compared to full tensor and monte carlo approaches

a new perspective on functional integration the core of this article is a general theorem with a large number of specializations given a manifold n and a finite number of one parameter groups of point transformations on n with generators y x 1 x d we obtain via functional integration over spaces of pointed paths on n paths with one fixed point a one parameter group of functional operators acting on tensor or spinor fields on n the generator of this group is a quadratic form in the lie derivatives lx in the x direction plus a term linear in ly the basic functional integral is over l2 1 paths x t n continuous paths with square integrable first derivative although the integrator is invariant under time translation the integral is powerful enough to be used for systems which are not time translation invariant seven nontrivial applications of the basic formula are given and the semiclassical expansion is computed the methods of proof are rigorous and combine albeverio ho egh krohn oscillatory integrals with elworthy s parametr

efficient posterior exploration of a high dimensional groundwater model from two stage markov chain monte carlo simulation and polynomial chaos expansion 1 this study reports on two strategies for accelerating posterior inference of a highly parameterized and cpu demanding groundwater flow model our method builds on previous stochastic collocation approaches e g marzouk and xiu 2009 and marzouk and najm 2009 and uses generalized polynomial chaos gpc theory and dimensionality reduction to emulate the output of a large scale groundwater flow model the resulting surrogate model is cpu efficient and serves to explore the posterior distribution at a much lower computational cost using two stage mcmc simulation the case study reported in this paper demonstrates a two to five times speed up in sampling efficiency

a robust approach to independent component analysis of signals with high level noise measurements we propose a robust approach for independent component analysis ica of signals where observations are contaminated with high level additive noise and or outliers the source signals may contain mixtures of both sub gaussian and super gaussian components and the number of sources is unknown our robust approach includes two procedures in the first procedure a robust prewhitening technique is used to reduce the power of additive noise the dimensionality and the correlation among sources a cross validation technique is introduced to estimate the number of sources in this first procedure in the second procedure a nonlinear function is derived using the parameterized t distribution density model this nonlinear function is robust against the undue influence of outliers fundamentally moreover the stability of the proposed algorithm and the robust property of misestimating the parameters kurtosis have been studied by combining the t distribution model with a family of light tailed distributions sub gaussian model we can separate the mixture of sub gaussian and super gaussian source components through the analysis of artificially synthesized data and real world magnetoencephalographic meg data we illustrate the efficacy of this robust approach

dimensionality reduction via sparse support vector machines we describe a methodology for performing variable ranking and selection using support vector machines svms the method constructs a series of sparse linear svms to generate linear models that can generalize well and uses a subset of nonzero weighted variables found by the linear models to produce a final nonlinear model the method exploits the fact that a linear svm no kernels with l1 norm regularization inherently performs variable selection as a side effect of minimizing capacity of the svm model the distribution of the linear model weights provides a mechanism for ranking and interpreting the effects of variables starplots are used to visualize the magnitude and variance of the weights for each variable we illustrate the effectiveness of the methodology on synthetic data benchmark problems and challenging regression problems in drug design this method can dramatically reduce the number of variables and outperforms svms trained using all attributes and using the attributes selected according to correlation coefficients the visualization of the resulting models is useful for understanding the role of underlying variables

a domain decomposition method of stochastic pdes recent advances in high performance computing systems and sensing technologies motivate computational simulations with extremely high resolution models with capabilities to quantify uncertainties for credible numerical predictions a two level domain decomposition method is reported in this investigation to devise a linear solver for the large scale system in the galerkin spectral stochastic finite element method ssfem in particular a two level scalable preconditioner is introduced in order to iteratively solve the large scale linear system in the intrusive ssfem using an iterative substructuring based domain decomposition solver the implementation of the algorithm involves solving a local problem on each subdomain that constructs the local part of the preconditioner and a coarse problem that propagates information globally among the subdomains the numerical and parallel scalabilities of the two level preconditioner are contrasted with the previously developed one level preconditioner for two dimensional flow through porous media and elasticity problems with spatially varying non gaussian material properties a distributed implementation of the parallel algorithm is carried out using mpi and petsc parallel libraries the scalabilities of the algorithm are investigated in a linux cluster

on decompositions of multivariate functions we present formulas that allow us to decompose a function f of d variables into a sum of 2 d terms fu indexed by subsets u of f1 dg where each term fu depends only on the variables with indices in u the decomposition depends on the choice of d commuting projections fpjg d 1 where pj f does not depend on the variable xj we present an explicit formula for fu which is new even for the anova and anchored decompositions both are special cases of the general decomposition we show that the decomposition is minimal in the following sense if f is expressible as a sum in which there is no term that depends on all of the variables indexed by the subset z then for every choice of fpjg d 1 the terms fu 0 for all subsets u containing z furthermore in a reproducing kernel hilbert space setting we give su cient conditions for the terms fu to be mutually orthogonal

cross validatory choice of the number of components from a principal component analysis a method is described for choosing the number of components to retain in a principal component analysis when the aim is dimensionality reduction the correspondence between principal component analysis and the singular value decomposition of the data matrix is used the method is based on successively predicting each element in the data matrix after deleting the corresponding row and column of the matrix and makes use of recently published algorithms for updating a singular value decomposition these are very fast which renders the proposed technique a practicable one for routine data analysis

weighted classifier ensemble based on quadratic form diversity and accuracy are the two key factors that decide the ensemble generalization error constructing a good ensemble method by balancing these two factors is difficult because increasing diversity is at the cost of reducing accuracy normally in order to improve the performance of an ensemble while avoiding the difficulty derived of balancing diversity and accuracy we propose a novel method that weights each classifier in the ensemble by maximizing three different quadratic forms in this paper the optimal weight of individual classifiers is obtained by minimizing the ensemble error rather than analyzing diversity and accuracy since it is difficult to minimize the general form of the ensemble error directly we approximate the error in an objective function subject to two constraints w i 1 and 1 w i 1 particularly we introduce an error term with a weight vector w0 and subtract this error with the quadratic form to obtain our approximated error this subtraction makes minimizing the approximation form equivalent to maximizing the original quadratic form theoretical analysis finds that when the value of the quadratic form is maximized the error of an ensemble system with the corresponding optimal weight w will be smallest especially compared with the ensemble with w0 finally we demonstrate improved classification performance from the experimental results of an artificial dataset uci datasets and polsar image data a new weighted classifier ensemble method is proposed an approximation form of the ensemble error is introduced an optimal weight vector is sought based on minimizing the approximation form it is converted into maximizing quadratic forms by invoking a known weight vector the larger the value of the quadratic form is the lower the ensemble error is

a stochastic collocation method for elliptic partial differential equations with random input data in this paper we propose and analyze a stochastic collocation method to solve elliptic partial differential equations with random coefficients and forcing terms input data of the model the input data are assumed to depend on a finite number of random variables the method consists in a galerkin approximation in space and a collocation in the zeros of suitable tensor product orthogonal polynomials gauss points in the probability space and naturally leads to the solution of uncoupled deterministic problems as in the monte carlo approach it can be seen as a generalization of the stochastic galerkin method proposed in i babuscka r tempone and g e zouraris siam j numer anal 42 2004 pp 800 825 and allows one to treat easily a wider range of situations such as input data that depend nonlinearly on the random variables diffusivity coefficients with unbounded second moments and random variables that are correlated or even unbounded we provide a rigorous convergence analysis and demonstrate exponential convergence of the probability error with respect to the number of gauss points in each direction in the probability space under some regularity assumptions on the random input data numerical examples show the effectiveness of the method

a new algorithm and worst case complexity for feynman kac path integration abstract we study algorithms for approximation of feynman kac path integrals in the worst case setting the algorithms use a finite number of samples of the initial condition and potential functions we present a new algorithm and an explicit bound on its cost to compute an e approximation to the feynman kac path integral we also establish bounds on the worst case complexity of feynman kac path integration the upper bound is equal to the cost of the new algorithm and is given in terms of the complexity of a certain function approximation problem the lower bound is given in terms of the complexity of a certain weighted integration problem for some classes of functions these two bounds coincide modulo a multiplicative factor in this case the new algorithm is almost optimal the new algorithm requires precomputation of some real coefficients that are combinations of multivariate integrals with special weights this precomputation is difficult and limits the application of the new algorithm we report the results of the precomputation for specific cases

influence in principal components analysis summary in linear regression the theoretical influence function and the various sample versions of it have an established place as diagnostic tools these same functions are developed here to provide methods for the detection of influential observations in principal components analysis the perturbation theory of real symmetric matrices unifies this development some interesting points of contrast with the regression case are noted and explained theoretically

linear programming support vector machines abstract based on the analysis of the conclusions in the statistical learning theory especially the vc dimension of linear functions linear programming support vector machines or svms are presented including linear programming linear and nonlinear svms in linear programming svms in order to improve the speed of the training time the bound of the vc dimension is loosened properly simulation results for both artificial and real data show that the generalization performance of our method is a good approximation of svms and the computation complex is largely reduced by our method

quasi monte carlo finite element methods for a class of elliptic partial differential equations with random coefficients in this paper quasi monte carlo qmc methods are applied to a class of elliptic partial differential equations pdes with random coefficients where the random coefficient is parametrized by a co

a note on the limiting procedures for path integrals limiting procedures for the feynman type path integral are considered the evolution operator is approximated with operators corresponding to the exponential of the hamiltonian s symbol the proof of convergence uses chernoff s theory and its extension on the class of stable operators the approach is especially adequate for the generalized coherent states path integral where decomposition of a hamiltonian into the sums used in the approach based on trotter s formula may be unnatural