generalized multiple kernel learning with data dependent priors multiple kernel learning mkl and classifier ensemble are two mainstream methods for solving learning problems in which some sets of features views are more informative than others or the features views within a given set are inconsistent in this paper we first present a novel probabilistic interpretation of mkl such that maximum entropy discrimination with a noninformative prior over multiple views is equivalent to the formulation of mkl instead of using the noninformative prior we introduce a novel data dependent prior based on an ensemble of kernel predictors which enhances the prediction performance of mkl by leveraging the merits of the classifier ensemble with the proposed probabilistic framework of mkl we propose a hierarchical bayesian model to learn the proposed data dependent prior and classification model simultaneously the resultant problem is convex and other information e g instances with either missing views or missing labels can be seamlessly incorporated into the data dependent priors furthermore a variety of existing mkl models can be recovered under the proposed mkl framework and can be readily extended to incorporate these priors extensive experiments demonstrate the benefits of our proposed framework in supervised and semisupervised settings as well as in tasks with partial correspondence among multiple views

lle score a new filter based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition the task of feature selection is to find the most representative features from the original high dimensional data because of the absence of the information of class labels selecting the appropriate features in unsupervised learning scenarios is much harder than that in supervised scenarios in this paper we investigate the potential of locally linear embedding lle which is a popular manifold learning method in feature selection task it is straightforward to apply the idea of lle to the graph preserving feature selection framework however we find that this straightforward application suffers from some problems for example it fails when the elements in the feature are all equal it does not enjoy the property of scaling invariance and cannot capture the change of the graph efficiently to solve these problems we propose a new filter based feature selection method based on lle in this paper which is named as lle score the proposed criterion measures the difference between the local structure of each feature and that of the original data our experiments of classification task on two face image data sets an object image data set and a handwriting digits data set show that lle score outperforms state of the art methods including data variance laplacian score and sparsity score

statistical fitting and validation of non linear simulation metamodels a case study abstract linear regression metamodels have been widely used to explain the behavior of computer simulation models although they do not always provide a good global fit to smooth response functions of arbitrary shape in the case study discussed in this paper the use of several linear regression polynomial results in a poor fit the use of a non linear regression metamodeling methodology provides simple functions that adequately approximate the behavior of the target simulation model the importance of metamodel validation is emphasized by using the generalization of rao s test to non linear metamodels and double cross validation

consistency of spectral clustering consistency is a key property of all statistical procedures analyzing randomly sampled data surprisingly despite decades of work little is known about consistency of most clustering algorithms in this paper we investigate consistency of the popular family of spectral clustering algorithms which clusters the data with the help of eigenvectors of graph laplacian matrices we develop new methods to establish that for increasing sample size those eigenvectors converge to the eigenvectors of certain limit operators as a result we can prove that one of the two major classes of spectral clustering normalized clustering converges under very general conditions while the other unnormalized clustering is only consistent under strong additional assumptions which are not always satisfied in real data we conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

multiresponse robust design mean square error mse criterion abstract most of the published literature on robust design methodology is basically concerned with optimization of a single response or quality characteristic which is often most critical to consumers however manufactured products are typically characterized by numerous quality characteristics in this paper we present a general framework for the multivariate problem when data are collected from a combined array within the framework a mean square error mse criterion is utilized and a non linear multiobjective programming problem based on the individual mse functions of each response is proposed for quality improvement we adapted a suitable non linear optimization algorithm to solve the proposed formulation the optimization method used in this paper generates a string of solutions called pareto optimal solutions rather than a one shot optimum solution to make selections and evaluate the trade offs the paper also presents an example and comparative results in order to demonstrate the potentials of the proposed approach

unified framework for representing and ranking in the database retrieval and nearest neighbor classification tasks the two basic problems are to represent the query and database objects and to learn the ranking scores of the database objects to the query many studies have been conducted for the representation learning and the ranking score learning problems however they are always learned independently from each other in this paper we argue that there are some inner relationships between the representation and ranking of database objects and try to investigate their relationships by learning them in a unified way to this end we proposed the unified framework for representation and ranking ur2 of objects for the database retrieval and nearest neighbor classification tasks the learning of representation parameter and the ranking scores are modeled within one single unified objective function the objective function is optimized alternately with regard to representation parameter and the ranking scores based on the optimization results iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way moreover with two different formulas of representation feature selection and subspace learning we give two versions of ur2 the proposed algorithms are tested on two challenging tasks mri image based brain tumor retrieval and nearest neighbor classification based protein identification the experiments show the advantage of the proposed unified framework over the state of the art independent representation and ranking methods highlightsobject representation and ranking are two basic problems for database retrieval they are always studies separately we proposed the first unified framework for both representation and ranking experimental results show its advantage over independent representation or ranking methods

restricted subset selection procedures for simulation this paper develops two procedures for screening a set of normal populations with unknown moments in order that the final subset of selected populations satisfies the following requirements involving the user specified parameters p and m with probability at least p the selected subset will contain a population whose mean lies less than the distance from the largest mean although the size of the selected subset is random at most m populations will finally be chosen where m is usually taken small enough to reserve adequate resources for a more intensive follow up study the exact procedure ve is a two stage random sampling scheme that is designed to compare transient or steady state simulation models based on independent replications of each model the heuristic procedure vs is a modification of ve that is designed to compare steady state simulation models based on a single prolonged run of each model a complete justification is given for ve together with tables of the constants required by bo

an explicit expression for the harish chandra c function for the homogeneous space so 2 n s o 2 o n n 2 4 6 this paper is devoted to presenting an explicit expression for the harish chandra c function for the lie ball our computation technique is based on the expression for the generalized spherical function and its asymptotic behavior

active learning based on minimization of the expected path length of random walks on the learned manifold structure abstract active learning algorithms aim at selecting important samples to label for subsequent machine learning tasks many active learning algorithms make use of the reproducing kernel hilbert space rkhs induced by a gaussian radial basis function rbf kernel and leverage the geometrical structure of the data for query sample selection parameters for the kernel function and the k nearest neighborhood graph must be properly set beforehand as a tool exploring the structure of data active learning algorithms with automatic tuning of those parameters are desirable in this paper local linear embedding lle with convex constraints on neighbor weights is used to learn the geometrical structure of the data in the rkhs induced by a gaussian rbf kernel automatic tuning of the kernel parameter is based on the assumption that the geometrical structure of the data in the rkhs is sparse and local with the markov matrix established based on the learned lle weight matrix the total expected path length of the random walks from all samples to selected samples is proposed to be a criterion for query sample selection a greedy algorithm having a guaranteed solution bound is developed to select query samples and a two phase scheme is also proposed for scaling the proposed active learning algorithm experimental results on data sets including hundreds to tens of thousands of samples have shown the feasibility of the proposed approach

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

minimizing the bias and variance of the gradient estimate in rsm simulation studies abstract to minimize the variance of the response estimator in response surface methodology rsm based simulations most users extend the design points to the limits of the subregion under investigation our analysis however shows that when common pseudorandom numbers are used this practice likely increases the variance of the estimated gradient under the criterion of the variance of the gradient and the mahalanobis distance function we show that smaller subregions reduce both the bias and the variance of the estimated gradient furthermore we discuss how users may take advantage of the magnitude of the induced correlations to draw inferences about the search process and to manipulate the subregion size our analysis offers a new direction for future research

l2 poisson integral representations of solutions of the hua system on the bounded symmetric domain su n n s u n u n abstract let d z m n c i n zz positive definite be the matrix ball of rank n and let hd be the associated hua operator for a complex number such that rei n 1 we give a necessary and sufficient condition on solutions f of the following hua system of differential equations on d h d f z 2 n 2 f z i n to have an l2 poisson integral representations over the shilov boundary s of d namely f is the poisson integral of an l2 function on the shilov boundary s if and only if its satisfies the following growth condition of hardy type f 2 sup 0 r 1 r 2 n n rei s f ru 2 du in particular for in this shows that every harmonic function f with respect to the hua operator hd has an l2 poisson representation over s if and only if its hardy norm is finite

global optimization based on novel heuristics low discrepancy sequences and genetic algorithms abstract in this paper a new heuristic hybrid technique for bound constrained global optimization is proposed we developed iterative algorithm called glp s that uses genetic algorithms lp low discrepancy sequences of points and heuristic rules to find regions of attraction when searching a global minimum of an objective function subsequently nelder mead simplex local search technique is used to refine the solution the combination of the three techniques genetic algorithms lp o low discrepancy search and simplex search provides a powerful hybrid heuristic optimization method which is tested on a number of benchmark multimodal functions with 10 150 dimensions and the method properties applicability convergence consistency and stability are discussed in detail

genetic algorithm based technique for solving chance constrained problems management and measurement of risk is an important issue in almost all areas that require decisions to be made under uncertain information chance constrained programming ccp have been used for modelling and analysis of risks in a number of application domains however the resulting mathematical problems are non trivial to represent using algebraic modelling languages and pose significant computational challenges due to their non linear non convex and the stochastic nature we develop and implement c classes to represent such ccp problems we propose a framework consisting of genetic algorithm and monte carlo simulation in order to process the problems the non linear and non convex nature of the ccp problems are processed using genetic algorithm whereas the stochastic nature is addressed through simulation the computational investigations have shown that the framework can efficiently represent and obtain good solutions for seven test problems

hua operators on bounded homogeneous domains in cnand alternative reproducing kernels for holomorphic functions let d be a bounded homogeneous domain in cn note that d is not assumed to be hermitian symmetric in this work we are interested in studying various classes of harmonic functions on d and the possibility of representing them as poisson integrals over the bergman shilov boundary one such class of harmonic functions is the hua harmonic functions specifically by forming a contraction of with the holomorphic curvature tensor we define a canonical system of differential operators which generalizes the classical hua system this system is invariant under all bi holomorphisms of d the hua harmonic functions are by definition the nullspace of this system our main result concerning this system is that every bounded hua harmonic function is the poisson integral over the bergman shilov boundary of a uniquel function against the poisson kernel for the laplace beltrami operator we also consider spaces of harmonic functions defined as the kernel of a single real differential operator which is invariant under a particular solvable lie group which acts transitively on d we show that there exists such an operator which a annihilates holomorphic functions b satisfies the hormander condition and c has the bergman shilov boundary as its maximal boundary it follows that the corresponding bounded harmonic functions are in one to one correspondence with thel functions on the bergman shilov boundary under poisson integration

discriminative semi supervised feature selection via manifold regularization feature selection has attracted a huge amount of interest in both research and application communities of data mining we consider the problem of semi supervised feature selection where we are given a small amount of labeled examples and a large amount of unlabeled examples since a small number of labeled samples are usually insufficient for identifying the relevant features the critical problem arising from semi supervised feature selection is how to take advantage of the information underneath the unlabeled data to address this problem we propose a novel discriminative semi supervised feature selection method based on the idea of manifold regularization the proposed approach selects features through maximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data in comparison with previous semi supervised feature selection algorithms our proposed semi supervised feature selection method is an embedded feature selection method and is able to find more discriminative features we formulate the proposed feature selection method into a convex concave optimization problem where the saddle point corresponds to the optimal solution to find the optimal solution the level method a fairly recent optimization method is employed we also present a theoretic proof of the convergence rate for the application of the level method to our problem empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi supervised feature selection method

validation of regression metamodels in simulation bootstrap approach abstract simulation experiments are often analyzed through a linear regression model of their input output data such an analysis yields a metamodel or response surface for the underlying simulation model this metamodel can be validated through various statistics this article studies 1 the coefficient of determination r square for generalized least squares and 2 a lack of fit f statistic originally formulated by rao biometrika 46 1959 49 who assumed multivariate normality to derive the distributions of these two validation statistics this paper shows how to apply bootstrapping without assuming normality to illustrate the performance of these bootstrapped validation statistics the paper uses monte carlo experiments with simple models for these models i r square is a conservative statistic rejecting a valid metamodel relatively rarely so its power is low ii rao s original statistic may reject a valid metamodel too often iii bootstrapping rao s statistic gives only slightly conservative results so its power is relatively high

an optimal algorithm for approximate nearest neighbor searching fixed dimensions consider a set of s of n data points in real d dimensional space r d where distances are measured using any minkowski metric in nearest neighbor searching we preprocess s into a data structure so that given any query point q r d is the closest point of s to q can be reported quickly given any positive real e data point p is a 1 e approximate nearest neighbor of q if its distance from q is within a factor of 1 e of the distance to the true nearest neighbor we show that it is possible to preprocess a set of n points in r d in o dn log n time and o dn space so that given a query point q r d and e 0 a 1 e approximate nearest neighbor of q can be computed in o c d e log n time where c d e d 1 6d e d is a factor depending only on dimension and e in general we show that given an integer k 1 1 e approximations to the k nearest neighbors of q can be computed in additional o kd log n time

simulation designs for quadratic response surface models in the presence of model misspecification this article considers the selection of experimental designs for the estimation of second order response surface metamodels in a simulation environment rather than construct designs based on the premise that the postulated model exactly represents the simulated response as is the case in optimal design theory we assume that the estimation process may be biased by the presence of unfitted third order terms we therefore seek to specify experimental plans that address the bias due to possible model misspecification as well as traditional variance considerations the performance measure used for this fit protect scenario is box and draper s design criterion of average mean squared error of predicted response four important classes of response surface experimental plans are examined 1 central composite designs 2 box behnken plans 3 three level factorial experiments and 4 small central composite designs each design class is studied in conjunction with three pseudorandom number assignment str

fisher regularized support vector machine this paper proposes fisher regularized support vector machine fishersvm fishersvm is a graph based supervised learning method fishersvm has two regularization terms maximum margin and fisher regularizations fishersvm aims to maximize the margin and minimize the within class scatter support vector machine svm and fisher discriminant analysis fda are two commonly used methods in machine learning and pattern recognition a combined method of the linear svm and fda called svm lda linear discriminant analysis has been proposed only for the linear case this paper generalizes this combined method to the nonlinear case from the view of regularization a fisher regularization is defined and incorporated into svm to obtain a fisher regularized support vector machine fishersvm in fishersvm there are two regularization terms the maximum margin regularization and fisher regularization which allow fishersvm to maximize the classification margin and minimize the within class scatter roughly speaking fishersvm can approximatively fulfill the fisher criterion and obtain good statistical separability this paper also discusses the connections of fishersvm to graph based regularization methods and the mathematical programming method to kernel fisher discriminant analysis kfda it shows that fishersvm can be thought of as a graph based supervised learning method or a robust kfda experimental results on artificial and real world data show that fishersvm has a promising generalization performance

regression metamodels for simulation with common random numbers comparison of validation tests and confidence intervals linear regression analysis is important in many fields in the analysis of simulation results a regression metamodel can be applied even when common pseudorandom numbers are used to test the validity of the specified regression model rao 1959 generalized the f statistic for lack of fit whereas kleijnen 1983 proposed a cross validation procedure using a student s t statistic combined with bonferroni s inequality this paper reports on an extensive monte carlo experiment designed to compare these two methods under the normality assumption cross validation is conservative whereas rao s test realizes its nominal type i error and has high power robustness is investigated through lognormal and uniform distributions when simulation responses are distributed lognormally then cross validation using ordinary least squares is the only technique that has acceptable type i error uniform distributions give results similar to the normal case once the regression model is validated confidence intervals for the individual regression parameters are computed the monte carlo experiment compares several confidence interval procedures under normality rao s procedure is preferred since it has good coverage probability and acceptable half length under lognormality ordinary least squares achieves nominal coverage probability uniform distributions again give results similar to the normal case

local linear regression on manifolds and its geometric interpretation high dimensional data analysis has been an active area and the main focus areas have been variable selection and dimension reduction in practice it occurs often that the variables are located on an unknown lower dimensional nonlinear manifold under this manifold assumption one purpose of this article is regression and gradient estimation on the manifold and another is developing a new tool for manifold learning as regards the first aim we suggest directly reducing the dimensionality to the intrinsic dimension d of the manifold and performing the popular local linear regression llr on a tangent plane estimate an immediate consequence is a dramatic reduction in the computational time when the ambient space dimension p d we provide rigorous theoretical justification of the convergence of the proposed regression and gradient estimators by carefully analyzing the curvature boundary and nonuniform sampling effects we propose a bandwidth selector that can handle heteroscedastic errors with re

small composite designs small second order composite designs were suggested by hartley 1959 westlake 1965 provided even smaller designs for k 5 7 and 9 factors for which intricate construction methods were needed here simple designs formed using plackett and burman 1946 designs are offered for k 5 7 and 9 designs with one run fewer than westlake s for k 5 and 7 and three fewer for k 9 are feasible by deleting repeat points that occur in some of the designs

active learning based on locally linear reconstruction we consider the active learning problem which aims to select the most representative points out of many existing active learning techniques optimum experimental design oed has received considerable attention recently the typical oed criteria minimize the variance of the parameter estimates or predicted value however these methods see only global euclidean structure while the local manifold structure is ignored for example i optimal design selects those data points such that other data points can be best approximated by linear combinations of all the selected points in this paper we propose a novel active learning algorithm which takes into account the local structure of the data space that is each data point should be approximated by the linear combination of only its neighbors given the local reconstruction coefficients for every data point and the coordinates of the selected points a transductive learning algorithm called locally linear reconstruction llr is proposed to reconstruct every other point the most representative points are thus defined as those whose coordinates can be used to best reconstruct the whole data set the sequential and convex optimization schemes are also introduced to solve the optimization problem the experimental results have demonstrated the effectiveness of our proposed method

analyzing simulation experiments with common random numbers to analyze simulation runs which use the same random numbers the blocking concept of experimental design is not needed instead this paper applies a linear regression model with a nondiagonal covariance matrix this covariance matrix does not need to have a specific pattern such as constant covariances a simple example yields surprising results the paper proposes a new framework for the error analysis this framework consists of three factors namely common random numbers replication model validity each with three levels

image interpolation via graph based bayesian label propagation in this paper we propose a novel image interpolation algorithm via graph based bayesian label propagation the basic idea is to first create a graph with known and unknown pixels as vertices and with edge weights encoding the similarity between vertices then the problem of interpolation converts to how to effectively propagate the label information from known points to unknown ones this process can be posed as a bayesian inference in which we try to combine the principles of local adaptation and global consistency to obtain accurate and robust estimation specially our algorithm first constructs a set of local interpolation models which predict the intensity labels of all image samples and a loss term will be minimized to keep the predicted labels of the available low resolution lr samples sufficiently close to the original ones then all of the losses evaluated in local neighborhoods are accumulated together to measure the global consistency on all samples moreover a graph laplacian based manifold regularization term is incorporated to penalize the global smoothness of intensity labels such smoothing can alleviate the insufficient training of the local models and make them more robust finally we construct a unified objective function to combine together the global loss of the locally linear regression square error of prediction bias on the available lr samples and the manifold regularization term it can be solved with a closed form solution as a convex optimization problem experimental results demonstrate that the proposed method achieves competitive performance with the state of the art image interpolation algorithms

hessian eigenmaps locally linear embedding techniques for high dimensional data we describe a method for recovering the underlying parametrization of scattered data m i lying on a manifold m embedded in high dimensional euclidean space the method hessian based locally linear embedding derives from a conceptual framework of local isometry in which the manifold m viewed as a riemannian submanifold of the ambient euclidean space r n is locally isometric to an open connected subset of euclidean space r d because does not have to be convex this framework is able to handle a significantly wider class of situations than the original isomap algorithm the theoretical framework revolves around a quadratic form h f m h f m f dm defined on functions f m r here hf denotes the hessian of f and h f averages the frobenius norm of the hessian over m to define the hessian we use orthogonal coordinates on the tangent planes of m the key observation is that if m truly is locally isometric to an open connected subset of r d then h f has a d 1 dimensional null space consisting of the constant functions and a d dimensional space of functions spanned by the original isometric coordinates hence the isometric coordinates can be recovered up to a linear isometry our method may be viewed as a modification of locally linear embedding and our theoretical framework as a modification of the laplacian eigenmaps framework where we substitute a quadratic form based on the hessian in place of one based on the laplacian

network unfolding map by vertex edge dynamics modeling the emergence of collective dynamics in neural networks is a mechanism of the animal and human brain for information processing in this paper we develop a computational technique using distributed processing elements in a complex network which are called particles to solve semisupervised learning problems three actions govern the particles dynamics generation walking and absorption labeled vertices generate new particles that compete against rival particles for edge domination active particles randomly walk in the network until they are absorbed by either a rival vertex or an edge currently dominated by rival particles the result from the model evolution consists of sets of edges arranged by the label dominance each set tends to form a connected subnetwork to represent a data class although the intrinsic dynamics of the model is a stochastic one we prove that there exists a deterministic version with largely reduced computational complexity specifically with linear growth furthermore the edge domination process corresponds to an unfolding map in such way that edges stretch and shrink according to the vertex edge dynamics consequently the unfolding effect summarizes the relevant relationships between vertices and the uncovered data classes the proposed model captures important details of connectivity patterns over the vertex edge dynamics evolution in contrast to the previous approaches which focused on only vertex or only edge dynamics computer simulations reveal that the new model can identify nonlinear features in both real and artificial data including boundaries between distinct classes and overlapping structures of data

new semi supervised classification method based on modified cluster assumption the cluster assumption which assumes that similar instances should share the same label is a basic assumption in semi supervised classification learning and has been found very useful in many successful semi supervised classification methods it is rarely noticed that when the cluster assumption is adopted there is an implicit assumption that every instance should have a crisp class label assignment in real applications however there are cases where it is difficult to tell that an instance definitely belongs to one class and does not belong to other neighboring classes in such cases it is more adequate to assume that similar instances should share similar label memberships rather than sharing a crisp label assignment here label memberships can be represented as a vector where each element corresponds to a class and the value at the element expresses the likelihood of the concerned instance belonging to the class by adopting this modified cluster assumption in this paper we propose a new semi supervised classification method that is semi supervised classification based on class membership ssccm specifically we try to solve the decision function and adequate label memberships for instances simultaneously and constrain that an instance and its local weighted mean lwm share the same label membership vector where the lwm is a robust image of the instance constructed by calculating the weighted mean of its neighboring instances we formulate the problem in a unified objective function for the labeled unlabeled data and their lwms based on the square loss function and take an alternating iterative strategy to solve it in which each step generates a closed form solution and the convergence is guaranteed the solution will provide both the decision function and the label membership function for classification their classification results can verify each other and the reliability of semi supervised classification learning might be enhanced by checking the consistency between those two predictions experiments show that ssccm obtains encouraging results compared to state of the art semi supervised classification methods

a mean squared error criterion for the design of experiments in estimating a response surface over a design region of interest mean squared error can arise both from sampling variance and bias introduced by model inadequacy the criterion adopted here for experimental design attempts to protect against bias resulting from a large class of deviations from the assumed model two algorithms are proposed for the construction of either approximate or exact designs and an example is given

weighted multivariate mean square error for processes optimization a case study on flux cored arc welding for stainless steel claddings a mathematical programming technique developed recently that optimizes multiple correlated characteristics is the multivariate mean square error mmse the mmse approach has obtained noteworthy results by avoiding the production of inappropriate optimal points that can occur when a method fails to take into account a correlation structure where the mmse approach is deficient however is in cases where the multiple correlated characteristics need to be optimized with varying degrees of importance the mmse approach in treating all responses as having the same importance is unable to attribute the desired weights this paper thus introduces a strategy that weights the responses in the mmse approach the method called the weighted multivariate mean square error wmmse utilizes a weighting procedure that integrates principal component analysis pca and response surface methodology rsm in doing so wmmse obtains uncorrelated weighted objective functions from the original responses after being mathematically programmed these functions are optimized by employing optimization algorithms we applied wmmse to optimize a stainless steel cladding application executed via the flux cored arc welding fcaw process four input parameters and eight response variables were considered stainless steel cladding which carries potential benefits for a variety of industries takes low cost materials and deposits over their surfaces materials having anti corrosive properties optimal results were confirmed which ensured the deposition of claddings with defect free beads exhibiting the desired geometry and demonstrating good productivity indexes

response surface methodology s steepest ascent and step size revisited abstract response surface methodology rsm searches for the input combination maximizing the output of a real system or its simulation rsm is a heuristic that locally fits first order polynomials and estimates the corresponding steepest ascent sa paths however sa is scale dependent and its step size is selected intuitively to tackle these two problems this paper derives novel techniques combining mathematical statistics and mathematical programming technique 1 called adapted sa asa accounts for the covariances between the components of the estimated local gradient asa is scale independent the step size problem is solved tentatively technique 2 does follow the sa direction but with a step size inspired by asa mathematical properties of the two techniques are derived and interpreted numerical examples illustrate these properties the search directions of the two techniques are explored in monte carlo experiments these experiments show that in general asa gives a better search direction than sa

data visualization and dimensionality reduction using kernel maps with a reference point in this paper a new kernel based method for data visualization and dimensionality reduction is proposed a reference point is considered corresponding to additional constraints taken in the problem formulation in contrast with the class of kernel eigenmap methods the solution coordinates in the low dimensional space is characterized by a linear system instead of an eigenvalue problem the kernel maps with a reference point are generated from a least squares support vector machine ls svm core part that is extended with an additional regularization term for preserving local mutual distances together with reference point constraints the kernel maps possess primal and dual model representations and provide out of sample extensions e g for validation based tuning the method is illustrated on toy problems and real life data sets

simulation designs for the estimation of quadratic response surface gradients in the presence of model misspecification this article considers the construction of simulation designs for the ordinary least squares estimation of second order metamodels two premises underlie the development of these experimental strategies first it is assumed that the postulated metamodel may be misspecified due to the true model structure being of third order it is therefore important that the locations of the simulation experiments be specified to provide protection against bias as well as variance in the estimation of metamodel parameters the second premise is based on the observation that in many applications of metamodels functions of the fitted model coefficients such as the slope gradients are of greater interest than the response function the integrated mean squared error of slopes design criterion that is implemented here addresses both premises this criterion finds application in various optimum seeking methods and sensitivity analysis procedures combinations of four important classes of response surface designs and three pseudorandom number assignment strategies constitute the basis structure of the simulation designs studied the performance of these simulation designs is evaluated and subsequently compared to a similar set of experimental plans that have as their focus the estimation of the response function

semi supervised domain adaptation on manifolds in real life problems the following semi supervised domain adaptation scenario is often encountered we have full access to some source data which is usually very large the target data distribution is under certain unknown transformation of the source data distribution meanwhile only a small fraction of the target instances come with labels the goal is to learn a prediction model by incorporating information from the source domain that is able to generalize well on the target test instances we consider an explicit form of transformation functions and especially linear transformations that maps examples from the source to the target domain and we argue that by proper preprocessing of the data from both source and target domains the feasible transformation functions can be characterized by a set of rotation matrices this naturally leads to an optimization formulation under the special orthogonal group constraints we present an iterative coordinate descent solver that is able to jointly learn the transformation as well as the model parameters while the geodesic update ensures the manifold constraints are always satisfied our framework is sufficiently general to work with a variety of loss functions and prediction problems empirical evaluations on synthetic and real world experiments demonstrate the competitive performance of our method with respect to the state of the art

product and process yield estimation with gaussian quadrature gq reduction improvements over the gq full factorial approach abstract manufacturing or multivariate yield the fraction of unscreened products which conforms to all product specification limits is an important and commonly used metric for assessing and improving the quality of a production process current procedures for multivariate yield evaluation such as monte carlo simulation require substantial computing effort making the iterative adjustment of design parameters often impractical this paper introduces a new approach to multivariate yield evaluation based on a numerical integration procedure called gaussian quadrature reduction gqr the advantage of this approach is a large reduction in the computational burden associated with multivariate yield evaluation with virtually no loss in accuracy of the estimates the proposed procedure can be generalized to evaluate many other multivariate criteria such as expected costs and the desirability index the method is demonstrated for three yield evaluation test problems and comparisons to monte carlo based evaluations are presented

boundary value problems for the shilov boundary of a bounded symmetric domain of tube type abstract in this article the images of the poisson transform on the degenerate series representations attached to the shilov boundary of a bounded symmetric domain of tube type are considered we characterize the images by means of second order differential equations

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

simultaneous optimization of multiple responses represented by polynomial regression functions an algorithm is developed for the simultaneous optimization of several response functions that depend on the same set of controllable variables and are adequately represented by polynomial regression models of the same degree the data are first checked for linear dependencies among the responses if such dependencies exist a basic set of responses among which no linear functional relationships exist is chosen and used in developing a function that measures the distance of the vector of estimated responses from the estimated ideal optimum this distance function permits the user to account for the variances and covariances of the estimated responses and for the random error variation associated with the estimated ideal optimum suitable operating conditions for the simultaneous optimization of the responses are specified by minimizing the prescribed distance function over the experimental region an extension of the optimization procedure to mixture experiments is also given and the method is illustrat

from local similarities to global coding a framework for coding applications feature coding has received great attention in recent years as a building block of many image processing algorithms in particular the importance of the locality assumption in coding approaches has been studied in many previous works we review this assumption and claim that using the similarity of data points to a more global set of anchor points does not necessarily weaken the coding method as long as the underlying structure of the anchor points is considered we propose to capture the underlying structure by assuming a random walker over the anchor points we also show that our method is a fast approximation to the diffusion map kernel experiments on various data sets show that with a knowledge of the underlying structure of anchor points different state of the art coding algorithms may boost their performance in different learning tasks by utilizing the proposed method

simulation metamodel estimation using a combined correlation based variance reduction technique for first and higher order metamodels this paper develops a procedure for jointly applying the correlation based variance reduction techniques of common random numbers and control variates in a simulation experiment that is designed to estimate a linear metamodel that is linear in the regression model parameters for a single response variable expressed in terms of an input vector of design variables for the target system this procedure combines a the method of common random numbers for metamodel estimation and b a metamodel estimation scheme based on the method of control variates under specified conditions on the dependency structure of the simulation outputs and with respect to a variety of optimality criteria the combined procedure is shown to be superior to each of the following conventional correlation based variance reduction techniques independent random number streams common random number streams and control variates

simulation designs and correlation induction for reducing second order bias in first order response surfaces construction of simulation designs for the estimation of response surface metamodels is often based on optimal design theory underlying such designs is the assumption that the postulated model provides the correct representation of the simulated response as a result the location of design points and the assignment of pseudorandom number streams to these experiments are determined through the minimization of some function of the covariance matrix of the model coefficient estimators in contrast we assume that the postulated model may be incorrect attention is therefore directed to the development of simulation designs that offer protection against the bias due to possible model misspecification as well as error variance the particular situation examined is the estimation of first order response surface models in the presence of polynomials of order two traditional two level factorial plans combined with one of three pseudorandom number assignment strategies define the simulation designs specificati

function spaces and reproducing kernels on bounded symmetric domains there are several natural hilbert spaces of holomorphic functions on a bounded symmetric domain such are the bergman type spaces on which the holomorphic discrete series operates and the hardy type spaces which are related to the analytic continuation of the holomorphic discrete series also closely related is the bergmann space of entire functions on the ambient which arises as the closure of the polynomials with respect to a natural inner product the space of holomorphic polynomials decomposes into irreducible subspaces under the action of the isotropy group k of the domain the main facts about this decomposition were proved by schmid 21 for another proof see 22 each irreducible subspace contains a unique normalized l invariant spherical polynomial where l is the isotropy group of the shilov boundary in k our first main result is the explicit computation of the norms of the spherical polynomials with respect to each of the hilbert spaces considered for the domains of classical type they were considered by hua 9 and for some of the hilbert spaces this was done before by upmeier 24 using different methods for certain others there are partial results in 22 we are able to do this in a fairly simple unified way by making strong use of gindikin s generalized gamma function s next we obtain a description of the reproducing kernels of the k irreducible subspaces in each of our hilbert spaces and an expansion in terms of these for

perceptual multi channel visual feature fusion for scene categorization abstract effectively recognizing sceneries from a variety of categories is an indispensable but challenging technique in computer vision and intelligent systems in this work we propose a novel image kernel based on human gaze shifting aiming at discovering the mechanism of humans perceiving visually semantically salient regions within a scenery more specifically we first design a weakly supervised embedding algorithm which projects the local image features i e graphlets in this work onto the pre defined semantic space thereby we describe each graphlet by multiple visual features at both low level and high level it is generally acknowledged that humans attend to only a few regions within a scenery thus we formulate a sparsity constrained graphlet ranking algorithm which incorporates visual clues at both the low level and the high level according to human visual perception these top ranked graphlets are either visually or semantically salient we sequentially connect them into a path which mimics human gaze shifting lastly a so called gaze shifting kernel gsk is calculated based on the learned paths from a collection of scene images and a kernel svm is employed for calculating the scene categories comprehensive experiments on a series of well known scene image sets shown the competitiveness and robustness of our gsk we also demonstrated the high consistency of the predicted path with real human gaze shifting path

l p poisson integral representations of solutions of the hua system on hermitian symmetric spaces of tube type abstract in this paper we give a necessary and sufficient condition on eigenfunctions of the hua operator on a hermitian symmetric space of tube type x g k to have an l p poisson integral representations over the shilov boundary of x more precisely let c such that r 1 2 being the genus of x and let f be a c valued function on x satisfying the following hua system of second order differential equations h q f 2 2 32 2 f z then f has an l p poisson integral representation 1 p over the shilov boundary of x if and only if it satisfies the following growth condition of hardy type sup t 0 e r r t k f k a t p d k 1 p in particular for we obtain that a hua harmonic function on x has an l p poisson integral representation over the shilov boundary of x if and only if its hardy norm is finite

manifold regularization and semi supervised learning some theoretical analyses manifold regularization belkin et al 2006 is a geometrically motivated framework for machine learning within which several semi supervised algorithms have been constructed here we try to provide some theoretical understanding of this approach our main result is to expose the natural structure of a class of problems on which manifold regularization methods are helpful we show that for such problems no supervised learner can learn effectively on the other hand a manifold based learner that knows the manifold or learns it from unlabeled examples can learn with relatively few labeled examples our analysis follows a minimax style with an emphasis on finite sample results in terms of n the number of labeled examples these results allow us to properly interpret manifold regularization and related spectral and geometric algorithms in terms of their potential use in semi supervised learning

a variance minimization criterion to feature selection using laplacian regularization in many information processing tasks one is often confronted with very high dimensional data feature selection techniques are designed to find the meaningful feature subset of the original features which can facilitate clustering classification and retrieval in this paper we consider the feature selection problem in unsupervised learning scenarios which is particularly difficult due to the absence of class labels that would guide the search for relevant information based on laplacian regularized least squares which finds a smooth function on the data manifold and minimizes the empirical loss we propose two novel feature selection algorithms which aim to minimize the expected prediction error of the regularized regression model specifically we select those features such that the size of the parameter covariance matrix of the regularized regression model is minimized motivated from experimental design we use trace and determinant operators to measure the size of the covariance matrix efficient computational schemes are also introduced to solve the corresponding optimization problems extensive experimental results over various real life data sets have demonstrated the superiority of the proposed algorithms

multiclass data segmentation using diffuse interface methods on graphs we present two graph based algorithms for multiclass segmentation of high dimensional data on graphs the algorithms use a diffuse interface model based on the ginzburg landau functional related to total variation and graph cuts a multiclass extension is introduced using the gibbs simplex with the functional s double well potential modified to handle the multiclass case the first algorithm minimizes the functional using a convex splitting numerical scheme the second algorithm uses a graph adaptation of the classical numerical merriman bence osher mbo scheme which alternates between diffusion and thresholding we demonstrate the performance of both algorithms experimentally on synthetic data image labeling and several benchmark data sets such as mnist coil and webkb we also make use of fast numerical solvers for finding the eigenvectors and eigenvalues of the graph laplacian and take advantage of the sparsity of the matrix experiments indicate that the results are competitive with or better than the current state of the art in multiclass graph based segmentation algorithms for high dimensional data