generalized constraint neural network regression model subject to linear priors this paper is reports an extension of our previous investigations on adding transparency to neural networks we focus on a class of linear priors lps such as symmetry ranking list boundary monotonicity etc which represent either linear equality or linear inequality priors a generalized constraint neural network lps gcnn lps model is studied unlike other existing modeling approaches the gcnn lp model exhibits its advantages first any lp is embedded by an explicitly structural mode which may add a higher degree of transparency than using a pure algorithm mode second a direct elimination and least squares approach is adopted to study the model which produces better performances in both accuracy and computational cost over the lagrange multiplier techniques in experiments specific attention is paid to both hard strictly satisfied and soft weakly satisfied constraints for regression problems numerical investigations are made on synthetic examples as well as on the real world datasets simulation results demonstrate the effectiveness of the proposed modeling approach in comparison with other existing approaches

asymptotic expansions of lattice green s functions we derive asymptotic expansions for the green functions associated with coercive difference equations on general lattices these expansions lead to rigorous methods for approximating the lattice green functions by polyharmonic rational functions

cutting plane method for continuously constrained kernel based regression incorporating constraints into the kernel based regression is an effective means to improve regression performance nevertheless in many applications the constraints are continuous with respect to some parameters so that computational difficulties arise discretizing the constraints is a reasonable solution for these difficulties however in the context of kernel based regression most of existing works utilize the prior discretization strategy this strategy suffers from a few inherent deficiencies it cannot ensure that the regression result totally fulfills the original constraints and can hardly tackle high dimensional problems this paper proposes a cutting plane method cpm for constrained kernel based regression problems and a relaxed cpm r cpm for high dimensional problems the cpm discretizes the continuous constraints iteratively and ensures that the regression result strictly fulfills the original constraints for high dimensional problems the r cpm accepts a slight and controlled violation to attain a dimensional independent computational complexity the validity of the proposed methods is verified by numerical experiments

regression learning based on incomplete relationships between attributes abstract in recent years machine learning researchers have focused on methods to construct flexible and interpretable regression models however the method of obtaining complete knowledge from incomplete and fuzzy prior knowledge and the trade off between the generalization performance and the interpretability of the model are very important factors to consider in this paper we propose a new regression learning method complete relationships are obtained from the incomplete fuzzy relationships between attributes by using markov logic networks 29 the complete relationships are then applied to constrain the shape of the regression model in the optimization procedure to solve the trade off problem finally the benefits of our approach are illustrated on benchmark data sets and in real world experiments

a parallel fast multipole method for elliptic difference equations a new fast multipole formulation for solving elliptic difference equations on unbounded domains and its parallel implementation are presented these difference equations can arise directly in the description of physical systems e g crystal structures or indirectly through the discretization of pdes in the analog to solving continuous inhomogeneous differential equations using green s functions the proposed method uses the fundamental solution of the discrete operator on an infinite grid or lattice green s function fast solutions o n are achieved by using a kernel independent interpolation based fast multipole method unlike other fast multipole algorithms our approach exploits the regularity of the underlying cartesian grid and the efficiency of ffts to reduce the computation time our parallel implementation allows communications and computations to be overlapped and requires minimal global synchronization the accuracy efficiency and parallel performance of the method are demonstrated through numerical experiments on the discrete 3d poisson equation

an adaptive formally second order accurate version of the immersed boundary method like many problems in biofluid mechanics cardiac mechanics can be modeled as the dynamic interaction of a viscous incompressible fluid the blood and a visco elastic structure the muscular walls and the valves of the heart the immersed boundary method is a mathematical formulation and numerical approach to such problems that was originally introduced to study blood flow through heart valves and extensions of this work have yielded a three dimensional model of the heart and great vessels in the present work we introduce a new adaptive version of the immersed boundary method this adaptive scheme employs the same hierarchical structured grid approach but a different numerical scheme as the two dimensional adaptive immersed boundary method of roma et al a multilevel self adaptive version of the immersed boundary method ph d thesis courant institute of mathematical sciences new york university 1996 an adaptive version of the immersed boundary method j comput phys 153 2 1999 509 534 and is based on a formally second order accurate i e second order accurate for problems with sufficiently smooth solutions version of the immersed boundary method that we have recently described b e griffith c s peskin on the order of accuracy of the immersed boundary method higher order convergence rates for sufficiently smooth problems j comput phys 208 1 2005 75 105 actual second order convergence rates are obtained for both the uniform and adaptive methods by considering the interaction of a viscous incompressible flow and an anisotropic incompressible viscoelastic shell we also present initial results from the application of this methodology to the three dimensional simulation of blood flow in the heart and great vessels the results obtained by the adaptive method show good qualitative agreement with simulation results obtained by earlier non adaptive versions of the method but the flow in the vicinity of the model heart valves indicates that the new methodology provides enhanced boundary layer resolution differences are also observed in the flow about the mitral valve leaflets

support vector machines with constraints for sparsity in the primal parameters this paper introduces a new support vector machine svm formulation to obtain sparse solutions in the primal svm parameters providing a new method for feature selection based on svms this new approach includes additional constraints to the classical ones that drop the weights associated to those features that are likely to be irrelevant a svm formulation has been used where indicates the fraction of features to be considered this paper presents two versions of the proposed sparse classifier a 2 norm svm and a 1 norm svm the latter having a reduced computational burden with respect to the first one additionally an explanation is provided about how the presented approach can be readily extended to multiclass classification or to problems where groups of features rather than isolated features need to be selected the algorithms have been tested in a variety of synthetic and real data sets and they have been compared against other state of the art svm based linear feature selection methods such as 1 norm svm and doubly regularized svm the results show the good feature selection ability of the approaches

optimal multigrid convergence by elliptic hyperbolic splitting we describe a multigrid method for solving the steady euler equations that is optimal in the sense of requiring o n operations till convergence where n is the number of unknowns the method relies on an elliptic hyperbolic decomposition achieved by local preconditioning the splitting allows the embedded advection equations to be treated with streamwise semicoarsening rather than full coarsening which would not be effective a simple 2 d numerical computation is presented as proof of concept a convergence study indicates the split method has complexity o n over a wide range of grid spacings and mach numbers while the use of full coarsening for all equations makes the complexity deteriorate to almost o n1 5

a fast immersed boundary method for external incompressible viscous flows using lattice green s functions a new parallel computationally efficient immersed boundary method for solving three dimensional viscous incompressible flows on unbounded domains is presented immersed surfaces with prescribed motions are generated using the interpolation and regularization operators obtained from the discrete delta function approach of the original peskin s immersed boundary method unlike peskin s method boundary forces are regarded as lagrange multipliers that are used to satisfy the no slip condition the incompressible navier stokes equations are discretized on an unbounded staggered cartesian grid and are solved in a finite number of operations using lattice green s function techniques these techniques are used to automatically enforce the natural free space boundary conditions and to implement a novel block wise adaptive grid that significantly reduces the run time cost of solutions by limiting operations to grid cells in the immediate vicinity and near wake region of the immersed surface these techniques also enable the construction of practical discrete viscous integrating factors that are used in combination with specialized half explicit runge kutta schemes to accurately and efficiently solve the differential algebraic equations describing the discrete momentum equation incompressibility constraint and no slip constraint linear systems of equations resulting from the time integration scheme are efficiently solved using an approximation free nested projection technique the algebraic properties of the discrete operators are used to reduce projection steps to simple discrete elliptic problems e g discrete poisson problems that are compatible with recent parallel fast multipole methods for difference equations numerical experiments on low aspect ratio flat plates and spheres at reynolds numbers up to 3700 are used to verify the accuracy and physical fidelity of the formulation

fft fmm or multigrid a comparative study of state of the art poisson solvers for uniform and nonuniform grids in the unit cube from molecular dynamics and quantum chemistry to plasma physics and computational astrophysics poisson solvers in the unit cube are used in many applications in computational science and engineering in this work we benchmark and discuss the performance of the scalable methods for the poisson problem which are used widely in practice the fast fourier transform fft the fast multipole method fmm the geometric multigrid gmg and algebraic multigrid amg our focus is on solvers supporting high order highly nonuniform discretizations to allow comparisons with standard libraries we also compare adaptive solvers with solvers specialized for problems on regular grids that is fft and regular stencil multigrid since both are very popular algorithms for several practical applications for the multigrid we use the finite element variant of a high performance geometric multigrid hpgmg benchmark in total we compare five different codes three of which are developed in our group our fft gmg and

generalizing smoothness constraints from discrete samples we study how certain smoothness constraints for example piecewise continuity can be generalized from a discrete set of analog valued data by modifying the error backpropagation learning algorithm numerical simulations demonstrate that by imposing two heuristic objectives 1 reducing the number of hidden units and 2 minimizing the magnitudes of the weights in the network during the learning process one obtains a network with a response function that smoothly interpolates between the training data

nonlinear knowledge in kernel approximation prior knowledge over arbitrary general sets is incorporated into nonlinear kernel approximation problems in the form of linear constraints in a linear program the key tool in this incorporation is a theorem of the alternative for convex functions that converts nonlinear prior knowledge implications into linear inequalities without the need to kernelize these implications effectiveness of the proposed formulation is demonstrated on two synthetic examples and an important lymph node metastasis prediction problem all these problems exhibit marked improvements upon the introduction of prior knowledge over nonlinear kernel approximation approaches that do not utilize such knowledge

a fast lattice green s function method for solving viscous incompressible flows on unbounded domains a computationally efficient method for solving three dimensional viscous incompressible flows on unbounded domains is presented the method formally discretizes the incompressible navier stokes equations on an unbounded staggered cartesian grid operations are limited to a finite computational domain through a lattice green s function technique this technique obtains solutions to inhomogeneous difference equations through the discrete convolution of source terms with the fundamental solutions of the discrete operators the differential algebraic equations describing the temporal evolution of the discrete momentum equation and incompressibility constraint are numerically solved by combining an integrating factor technique for the viscous term and a half explicit runge kutta scheme for the convective term a projection method that exploits the mimetic and commutativity properties of the discrete operators is used to efficiently solve the system of equations that arises in each stage of the time integration scheme linear complexity fast computation rates and parallel scalability are achieved using recently developed fast multipole methods for difference equations the accuracy and physical fidelity of solutions are verified through numerical simulations of vortex rings

a practical bayesian framework for backpropagation networks a quantitative and practical bayesian framework is described for learning of mappings in feedforward networks the framework makes possible 1 objective comparisons between solutions using alternative network architectures 2 objective stopping rules for network pruning or growing procedures 3 objective choice of magnitude and type of weight decay terms or additive regularizers for penalizing large weights etc 4 a measure of the effective number of well determined parameters in a model 5 quantified estimates of the error bars on network parameters and on network output and 6 objective comparisons with alternative learning and interpolation models such as splines and radial basis functions the bayesian evidence automatically embodies occam s razor penalizing overflexible and overcomplex models the bayesian approach helps detect poor underlying assumptions in learning models for learning models well matched to a problem a good correlation between generalization ability and the bayesian evidence is obtained

a new multigrid approach to convection problems abstract a new multigrid method for convection problems is presented it is designed to overcome the problem of alignment the flow being aligned with the grid the technique employs semi coarsening in several directions simultaneously and gives rise to multiple coarser grids with the same total number of points per grid level but with different sizes in each co ordinate direction the amount of work per multigrid cycle is still o n as an example the method is applied to the nonlinear upwind differenced euler equations of gas dynamics in two dimensions convergence rates are estimated by two level fourier analysis for the linearised equations numerical experiments on the nonlinear equations confirm these estimates

a fast solver for poisson problems on infinite regular lattices the fast multipole method fmm provides a highly efficient computational tool for solving constant coefficient partial differential equations e g the poisson equation on infinite domains the solution to such an equation is given as the convolution between a fundamental solution and the given data function and the fmm is used to rapidly evaluate the sum resulting upon discretization of the integral this paper describes an analogous procedure for rapidly solving elliptic difference equations on infinite lattices in particular a fast summation technique for a discrete equivalent of the continuum fundamental solution is constructed the asymptotic complexity of the proposed method is o n s o u r c e where n s o u r c e is the number of points subject to body loads this is in contrast to fft based methods which solve a lattice poisson problem at a cost o n wlogn w independent of n s o u r c e where w is an artificial rectangular box containing the loaded points and n w is the number of lattice points in w

an adaptive version of the immersed boundary method a computational setting for the immersed boundary method employing an adaptive mesh refinement is presented enhanced accuracy for the method is attained locally by covering an immersed boundary vicinity with a sequence of nested progressively finer rectangular grid patches which dynamically follow the immersed boundary motion the set of equations describing the interaction between a non stationary viscous incompressible fluid and an immersed elastic boundary is solved by coupling a projection method specially designed for locally refined meshes to an implicit formulation of the immersed boundary method the main contributions of this work concern the formulation and the implementation of a multilevel self adaptive version of the immersed boundary method on locally refined meshes this approach is tested for a particular two dimensional model problem for which no significant difference is found between the solutions obtained on a mesh refined locally around the immersed boundary and on the associated uniform mesh built with the resolution of the finest level

a fast multi resolution lattice green s function method for elliptic difference equations abstract we propose a mesh refinement technique for solving elliptic difference equations on unbounded domains based on the fast lattice green s function flgf method the flgf method exploits the regularity of the cartesian mesh and uses the fast multipole method in conjunction with fast fourier transforms to yield linear complexity and decrease time to solution we extend this method to a multi resolution scheme and allow for locally refined cartesian blocks embedded in the computational domain appropriately chosen interpolation and regularization operators retain consistency between the discrete laplace operator and its inverse on the unbounded domain second order accuracy and linear complexity are maintained while significantly reducing the number of degrees of freedom and hence the computational cost

a generalized constraint neural network model associating partially known relationships for nonlinear regressions in an attempt to enhance the neural network technique so that it can evolve from a black box tool into a semi analytical one we propose a novel modeling approach of imposing generalized constraints on a standard neural network we redefine approximation problems by use of a new formalization with the aim of embedding prior knowledge explicitly into the model to the maximum extent a generalized constraint neural network gcnn model has therefore been developed which basically consists of two submodels one is constructed by the standard neural network technique to approximate the unknown part of the target function the other is formed from partially known relationships to impose generalized constraints on the whole model three issues arising after combination of the two submodels are discussed a the better approximation provided by the gcnn model compared with a standard neural network b the identifiability of parameters in the partially known relationships and c the discrepancy in the approximation due to removable singularities in the target function numerical studies of three benchmark problems show important findings that have not previously been reported in the literature significant benefits were observed from using the gcnn model in comparison with a standard neural network

design criteria for local euler preconditioning euler preconditioning has remarkable benefits in removing stiffness making systems of equations behave as a scalar equation preserving accuracy and decoupling the euler equations design criteria for optimal euler preconditioning are discussed that retain the basic preconditioning benefits and remove the causes of instabilities due to the use of preconditioning new families of 1d and 2d optimal euler preconditioners are presented that may satisfy the design criteria in an optimal way in particular focusing on resolution of the stability problem associated with stagnation points a stagnation preconditioner and a suboptimal van leer lee roe preconditioner are studied these preconditioners are less sensitive to flow angle variation across cells and or produce a closer to orthogonal eigenvector system

exact 1 norm support vector machines via unconstrained convex differentiable minimization support vector machines utilizing the 1 norm typically set up as linear programs mangasarian 2000 bradley and mangasarian 1998 are formulated here as a completely unconstrained minimization of a convex differentiable piecewise quadratic objective function in the dual space the objective function which has a lipschitz continuous gradient and contains only one additional finite parameter can be minimized by a generalized newton method and leads to an exact solution of the support vector machine problem the approach here is based on a formulation of a very general linear program as an unconstrained minimization problem and its application to support vector machine classification problems the present approach which generalizes both mangasarian 2004 and fung and mangasarian 2004 is also applied to nonlinear approximation where a minimal number of nonlinear kernel functions are utilized to approximate a function from a given number of function values

knowledge based kernel approximation prior knowledge in the form of linear inequalities that need to be satisfied over multiple polyhedral sets is incorporated into a function approximation generated by a linear combination of linear or nonlinear kernels in addition the approximation needs to satisfy conventional conditions such as having given exact or inexact function values at certain points determining such an approximation leads to a linear programming formulation by using nonlinear kernels and mapping the prior polyhedral knowledge in the input space to one defined by the kernels the prior knowledge translates into nonlinear inequalities in the original input space through a number of computational examples including a real world breast cancer prognosis dataset it is shown that prior knowledge can significantly improve function approximation

density estimation by mixture models with smoothing priors in the statistical approach for self organizing maps soms learning is regarded as an estimation algorithm for a gaussian mixture model with a gaussian smoothing prior on the centroid parameters the values of the hyperparameters and the topological structure are selected on the basis of a statistical principle however since the component selection probabilities are fixed to a common value the centroids concentrate on areas with high data density this deforms a coordinate system on an extracted manifold and makes smoothness evaluation for the manifold inaccurate in this article we study an extended som model whose component selection probabilities are variable to stabilize the estimation a smoothing prior on the component selection probabilities is introduced an estimation algorithm for the parameters and the hyperparameters based on empirical bayesian inference is obtained the performance of density estimation by the new model and the som model is compared via simulation experiments

parameter identifiability in statistical machine learning a review this review examines the relevance of parameter identifiability for statistical models used in machine learning in addition to defining main concepts we address several issues of identifiability closely related to machine learning showing the advantages and disadvantages of state of the art research and demonstrating recent progress first we review criteria for determining the parameter structure of models from the literature this has three related issues parameter identifiability parameter redundancy and reparameterization second we review the deep influence of identifiability on various aspects of machine learning from theoretical and application viewpoints in addition to illustrating the utility and influence of identifiability we emphasize the interplay among identifiability theory machine learning mathematical statistics information theory optimization theory information geometry riemann geometry symbolic computation bayesian inference algebraic geometry and others finally we present a new perspective together with the associated challenges

a flexible coefficient smooth transition time series model we consider a flexible smooth transition autoregressive star model with multiple regimes and multiple transition variables this formulation can be interpreted as a time varying linear model where the coefficients are the outputs of a single hidden layer feedforward neural network this proposal has the major advantage of nesting several nonlinear models such as the self exciting threshold autoregressive setar the autoregressive neural network ar nn and the logistic star models furthermore if the neural network is interpreted as a nonparametric universal approximation to any borel measurable function our formulation is directly comparable to the functional coefficient autoregressive far and the single index coefficient regression models a model building procedure is developed based on statistical inference arguments a monte carlo experiment showed that the procedure works in small samples and its performance improves as it should in medium size samples several real examples are also addressed

fast and accurate numerical methods for solving elliptic difference equations defined on lattices techniques for rapidly computing approximate solutions to elliptic pdes such as laplace s equation are well established for problems involving general domains and operators with constant coefficients a highly efficient approach is to rewrite the boundary value problem as a boundary integral equation bie and then solve the bie using fast methods such as e g the fast multipole method fmm the current paper demonstrates that this procedure can be extended to elliptic difference equations defined on infinite lattices or on finite lattice with boundary conditions of either dirichlet or neumann type as a representative model problem a lattice equivalent of laplace s equation on a square lattice in two dimensions is considered discrete analogs of bies are derived and fast solvers analogous to the fmm are constructed fast techniques are also constructed for problems involving lattices with inclusions and local deviations from perfect periodicity the complexity of the methods described is o n b o u n d a r y n s o u r c e n i n c where n b o u n d a r y is the number of nodes on the boundary of the domain n s o u r c e is the number of nodes subjected to body loads and n i n c is the number of nodes that deviate from perfect periodicity this estimate should be compared to the o n d o m a i nlogn d o m a i n estimate for fft based methods where n d o m a i n is the total number of nodes in the lattice so that in two dimensions n b o u n d a r y n d o m a i n 1 2 several numerical examples are presented

the evidence framework applied to classification networks three bayesian ideas are presented for supervised adaptive classifiers first it is argued that the output of a classifier should be obtained by marginalizing over the posterior distribution of the parameters a simple approximation to this integral is proposed and demonstrated this involves a moderation of the most probable classifier s outputs and yields improved performance second it is demonstrated that the bayesian framework for model comparison described for regression models in mackay 1992a b can also be applied to classification problems this framework successfully chooses the magnitude of weight decay terms and ranks solutions found using different numbers of hidden units third an information based data selection criterion is derived and demonstrated within this framework

dimensionality reduction via sparse support vector machines we describe a methodology for performing variable ranking and selection using support vector machines svms the method constructs a series of sparse linear svms to generate linear models that can generalize well and uses a subset of nonzero weighted variables found by the linear models to produce a final nonlinear model the method exploits the fact that a linear svm no kernels with l1 norm regularization inherently performs variable selection as a side effect of minimizing capacity of the svm model the distribution of the linear model weights provides a mechanism for ranking and interpreting the effects of variables starplots are used to visualize the magnitude and variance of the weights for each variable we illustrate the effectiveness of the methodology on synthetic data benchmark problems and challenging regression problems in drug design this method can dramatically reduce the number of variables and outperforms svms trained using all attributes and using the attributes selected according to correlation coefficients the visualization of the resulting models is useful for understanding the role of underlying variables

monotonic calibrated interpolated look up tables real world machine learning applications may have requirements beyond accuracy such as fast evaluation times and interpretability in particular guaranteed monotonicity of the learned function with respect to some of the inputs can be critical for user confidence we propose meeting these goals for low dimensional machine learning problems by learning flexible monotonic functions using calibrated interpolated look up tables we extend the structural risk minimization framework of lattice regression to monotonic functions by adding linear inequality constraints in addition we propose jointly learning interpretable calibrations of each feature to normalize continuous features and handle categorical or missing data at the cost of making the objective non convex we address large scale learning through parallelization mini batching and random sampling of additive regularizer terms case studies on real world problems with up to sixteen features and up to hundreds of millions of training samples demonstrate the proposed monotonic functions can achieve state of the art accuracy in practice while providing greater transparency to users

an immersed boundary method based on discrete stream function formulation for two and three dimensional incompressible flows an immersed boundary method is proposed in the framework of discrete stream function formulation for incompressible flows in order to impose the non slip boundary condition the forcing term is determined implicitly by solving a linear system the number of unknowns of the linear system is the same as that of the lagrangian points representing the body surface thus the extra cost in force calculation is negligible if compared with that in the basic flow solver in order to handle three dimensional flows at moderate reynolds numbers a parallelized flow solver based on the present method is developed using the domain decomposition strategy to verify the accuracy of the immersed boundary method proposed in this work flow problems of different complexity decaying vortices flows over stationary and oscillating cylinders and a stationary sphere and flow over low aspect ratio flat plate are simulated and the results are in good agreement with the experimental or computational data in previously published literatures

a kernel independent adaptive fast multipole algorithm in two and three dimensions we present a new fast multipole method for particle simulations the main feature of our algorithm is that it does not require the implementation of multipole expansions of the underlying kernel and it is based only on kernel evaluations instead of using analytic expansions to represent the potential generated by sources inside a box of the hierarchical fmm tree we use a continuous distribution of an equivalent density on a surface enclosing the box to find this equivalent density we match its potential to the potential of the original sources at a surface in the far field by solving local dirichlet type boundary value problems the far field evaluations are sparsified with singular value decomposition in 2d or fast fourier transforms in 3d we have tested the new method on the single and double layer operators for the laplacian the modified laplacian the stokes the modified stokes the navier and the modified navier operators in two and three dimensions our numerical results indicate that our method compares very well with the best known implementations of the analytic fmm method for both the laplacian and modified laplacian kernels its advantage is the relative simplicity of the implementation and its immediate extension to more general kernels

nonlinear knowledge based classification in this brief prior knowledge over general nonlinear sets is incorporated into nonlinear kernel classification problems as linear constraints in a linear program these linear constraints are imposed at arbitrary points not necessarily where the prior knowledge is given the key tool in this incorporation is a theorem of the alternative for convex functions that converts nonlinear prior knowledge implications into linear inequalities without the need to kernelize these implications effectiveness of the proposed formulation is demonstrated on publicly available classification data sets including a cancer prognosis data set nonlinear kernel classifiers for these data sets exhibit marked improvements upon the introduction of nonlinear prior knowledge compared to nonlinear kernel classifiers that do not utilize such knowledge

bayesian inference based on stationary fokker planck sampling a novel formalism for bayesian learning in the context of complex inference models is proposed the method is based on the use of the stationary fokker planck sfp approach to sample from the posterior density stationary fokker planck sampling generalizes the gibbs sampler algorithm for arbitrary and unknown conditional densities by the sfp procedure approximate analytical expressions for the conditionals and marginals of the posterior can be constructed at each stage of sfp the approximate conditionals are used to define a gibbs sampling process which is convergent to the full joint posterior by the analytical marginals efficient learning methods in the context of artificial neural networks are outlined offline and incremental bayesian inference and maximum likelihood estimation from the posterior are performed in classification and regression examples a comparison of sfp with other monte carlo strategies in the general problem of sampling from arbitrary densities is also presented it is shown that sfp is able to jump large low probability regions without the need of a careful tuning of any step size parameter in fact the sfp method requires only a small set of meaningful parameters that can be selected following clear problem independent guidelines the computation cost of sfp measured in terms of loss function evaluations grows linearly with the given model s dimension

optimal expected distance separating halfspace one recently proposed criterion to separate two data sets in discriminant analysis is to use a hyperplane which minimizes the sum of distances to it from all the misclassified data points here all distances are supposed to be measured by way of some fixed norm while misclassification means lying in the wrong halfspace in this paper we study the problem of determining such an optimal halfspace when points are distributed according to an arbitrary random vector x in rd in the unconstrained case in dimension d we prove that any optimal separating halfspace always balances the misclassified points moreover under polyhedrality assumptions on the support of x there always exists an optimal separating halfspace passing through d affinely independent points these results extend in a natural way when different norms or a fixed gauge are used to measure distances and we allow constraints modeling that certain points are forced to be correctly classified

the immersed boundary method a projection approach a new formulation of the immersed boundary method with a structure algebraically identical to the traditional fractional step method is presented for incompressible flow over bodies with prescribed surface motion like previous methods a boundary force is applied at the immersed surface to satisfy the no slip constraint this extra constraint can be added to the incompressible navier stokes equations by introducing regularization and interpolation operators the current method gives prominence to the role of the boundary force acting as a lagrange multiplier to satisfy the no slip condition this role is analogous to the effect of pressure on the momentum equation to satisfy the divergence free constraint the current immersed boundary method removes slip and non divergence free components of the velocity field through a projection the boundary force is determined implicitly without any constitutive relations allowing the present formulation to use larger cfl numbers compared to some past methods symmetry and positive definiteness of the system are preserved such that the conjugate gradient method can be used to solve for the flow field examples show that the current formulation achieves second order temporal accuracy and better than first order spatial accuracy in l 2 norms for one and two dimensional test problems results from two dimensional simulations of flows over stationary and moving cylinders are in good agreement with those from previous experimental and numerical studies

a regularized pairwise multi classification knowledge based machine and applications this paper presents a novel knowledge based linear classification model for multi category discrimination of sets or objects with prior knowledge the prior knowledge is in the form of multiple polyhedral sets belonging to one or more categories or classes and it is introduced as additional constraints into the formulation of the tikhonov linear least squares multi class support vector machine model the resulting formulation leads to a least squares problem that can be solved using matrix methods or iterative methods investigations include the development of a linear knowledge based classification model extended to the case of multi categorical discrimination and expressed as a single unconstrained optimization problem advantages of this formulation include explicit expressions for the classification weights of the classifier s and its ability to incorporate and handle prior knowledge directly to the classifiers in addition it can provide fast solutions to the optimal classification weights for multi categorical separation without the use of specialized solver software to evaluate the model data and prior knowledge from the wisconsin breast cancer prognosis and two phase flow regimes in pipes were used to train and test the proposed formulation

a group of knowledge incorporated multiple criteria linear programming classifiers classification is a main data mining task which aims at predicting the class label of new input data on the basis of a set of pre classified samples multiple criteria linear programming mclp is used as a classification method in the data mining area which can separate two or more classes by finding a discriminate hyperplane although mclp shows good performance in dealing with linear separable data it is no longer applicable when facing with nonlinear separable problems a kernel based multiple criteria linear programming kmclp model is developed to solve nonlinear separable problems in this method a kernel function is introduced to project the data into a higher dimensional space in which the data will have more chance to be linear separable kmclp performs well in some real applications however just as other prevalent data mining classifiers mclp and kmclp learn only from training examples in the traditional machine learning area there are also classification tasks in which data sets are classified only by prior knowledge i e expert systems some works combine the above two classification principles to overcome the faults of each approach in this paper we provide our recent works which combine the prior knowledge and the mclp or kmclp model to solve the problem when the input consists of not only training examples but also prior knowledge specifically how to deal with linear and nonlinear knowledge in mclp and kmclp models is the main concern of this paper numerical tests on the above models indicate that these models are effective in classifying data with prior knowledge

knowledge based linear programming we introduce a class of linear programs with constraints in the form of implications such linear programs arise in support vector machine classification where in addition to explicit datasets to be classified prior knowledge such as expert experience in the form of logical implications are imposed on the classifier the overall problem can be viewed either as a semi infinite linear program or as a linear program with equilibrium constraints which in either case can be solved by an equivalent simple linear program under mild assumptions