sparse ensembles using weighted combination methods based on linear programming an ensemble of multiple classifiers is widely considered to be an effective technique for improving accuracy and stability of a single classifier this paper proposes a framework of sparse ensembles and deals with new linear weighted combination methods for sparse ensembles sparse ensemble is to sparsely combine the outputs of multiple classifiers by using a sparse weight vector when the continuous outputs of multiple classifiers are provided in our methods the problem of solving sparse weight vector can be formulated as linear programming problems in which the hinge loss or and the 1 norm regularization are exploited both the hinge loss and the 1 norm regularization are techniques inducing sparsity used in machine learning we only ensemble classifiers with nonzero weight coefficients in these lp based methods the ensemble training error is minimized while the weight vector of ensemble learning is controlled which can be thought as implementing the structure risk minimization rule and naturally explains good performance of these methods the promising experimental results over uci data sets and the radar high resolution range profile data are presented

a solution algorithm for the fluid dynamic equations based on a stochastic model for molecular motion in this paper a stochastic model is presented to simulate the flow of gases which are not in thermodynamic equilibrium like in rarefied or micro situations for the interaction of a particle with others statistical moments of the local ensemble have to be evaluated but unlike in molecular dynamics simulations or dsmc no collisions between computational particles are considered in addition a novel integration technique allows for time steps independent of the stochastic time scale the stochastic model represents a fokker planck equation in the kinetic description which can be viewed as an approximation to the boltzmann equation this allows for a rigorous investigation of the relation between the new model and classical fluid and kinetic equations the fluid dynamic equations of navier stokes and fourier are fully recovered for small relaxation times while for larger values the new model extents into the kinetic regime numerical studies demonstrate that the stochastic model is consistent with navier stokes in that limit but also that the results become significantly different if the conditions for equilibrium are invalid the application to the knudsen paradox demonstrates the correctness and relevance of this development and comparisons with existing kinetic equations and standard solution algorithms reveal its advantages moreover results of a test case with geometrically complex boundaries are presented

an efficient particle fokker planck algorithm for rarefied gas flows this paper is devoted to the algorithmic improvement and careful analysis of the fokker planck kinetic model derived by jenny et al 1 and gorji et al 2 the motivation behind the fokker planck based particle methods is to gain efficiency in low knudsen rarefied gas flow simulations where conventional direct simulation monte carlo dsmc becomes expensive this can be achieved due to the fact that the resulting model equations are continuous stochastic differential equations in velocity space accordingly the computational particles evolve along independent stochastic paths and thus no collision needs to be calculated therefore the computational cost of the solution algorithm becomes independent of the knudsen number in the present study different computational improvements were persuaded in order to augment the method including an accurate time integration scheme local time stepping and noise reduction for assessment of the performance gas flow around a cylinder and lid driven cavity flow were studied convergence rates accuracy and computational costs were compared with respect to dsmc for a range of knudsen numbers from hydrodynamic regime up to above one in all the considered cases the model together with the proposed scheme give rise to very efficient yet accurate solution algorithms

ramp loss linear programming support vector machine the ramp loss is a robust but non convex loss for classification compared with other non convex losses a local minimum of the ramp loss can be effectively found the effectiveness of local search comes from the piecewise linearity of the ramp loss motivated by the fact that the l1 penalty is piecewise linear as well the l1 penalty is applied for the ramp loss resulting in a ramp loss linear programming support vector machine ramp lpsvm the proposed ramp lpsvm is a piecewise linear minimization problem and the related optimization techniques are applicable moreover the l1 penalty can enhance the sparsity in this paper the corresponding misclassification error and convergence behavior are discussed generally the ramp loss is a truncated hinge loss therefore ramp lpsvm possesses some similar properties as hinge loss svms a local minimization algorithm and a global search strategy are discussed the good optimization capability of the proposed algorithms makes ramp lpsvm perform well in numerical experiments the result of ramp lpsvm is more robust than that of hinge svms and is sparser than that of ramp svm which consists of the k penalty and the ramp loss

pedestrian detection in images via cascaded l1 norm minimization learning method a new cascaded l1 norm minimization learning clml method for pedestrian detection in images is proposed in this paper the proposed clml method which is designed from the perspective of vapnic s theory in the statistical learning integrates feature selection with classifier construction via solving meaningful optimization models the method incorporates three stages weak classifier learning strong classifier learning and the cascaded classifier construction in the weak classifier learning the l1 norm minimization learning lml and min max penalty function model are presented in the strong classifier learning an integer programming optimization model is built equaling the reformulation of lml in the integer space finally a cascade of lml classifiers is constructed to promote detection speed during the classifier learning and pedestrian detection histograms of oriented gradients of variable sized blocks v hog are used as feature descriptors experimental results on the inria and sdl human datasets show that the proposed method achieves a higher performance and speed than the state of the art methods

discrete velocity models and numerical schemes for the boltzmann bgk equation in plane and axisymmetric geometries we present new numerical models for computing transitional or rarefied gas flows as described by the boltzmann bgk and bgk es equations we first propose a new discrete velocity model based on the entropy minimization principle this model satisfies the conservation laws and the entropy dissipation moreover the problem of conservation and entropy for axisymmetric flows is investigated we find algebraic relations that must be satisfied by the discretization of the velocity derivative appearing in the transport operator then we propose some models that satisfy these constraints owing to these properties we obtain numerical schemes that are economic in terms of discretization and robust in particular we develop a linearized implicit scheme for computing stationary solutions of the discrete velocity bgk and bgk es models this scheme is the basis of a code which can compute high altitude hypersonic flows in 2d plane and axisymmetric geometries our results are analyzed and compared to other methods

margin maximizing feature elimination methods for linear and nonlinear kernel based discriminant functions feature selection for classification in high dimensional spaces can improve generalization reduce classifier complexity and identify important discriminating feature markers for support vector machine svm classification a widely used technique is recursive feature elimination rfe we demonstrate that rfe is not consistent with margin maximization central to the svm learning approach we thus propose explicit margin based feature elimination mfe for svms and demonstrate both improved margin and improved generalization compared with rfe moreover for the case of a nonlinear kernel we show that rfe assumes that the squared weight vector 2 norm is strictly decreasing as features are eliminated we demonstrate this is not true for the gaussian kernel and consequently rfe may give poor results in this case mfe for nonlinear kernels gives better margin and generalization we also present an extension which achieves further margin gains by optimizing only two degrees of freedom the hyperplane s intercept and its squared 2 norm with the weight vector orientation fixed we finally introduce an extension that allows margin slackness we compare against several alternatives including rfe and a linear programming method that embeds feature selection within the classifier design on high dimensional gene microarray data sets university of california at irvine uci repository data sets and alzheimer s disease brain image data mfe methods give promising results

consistent inflow and outflow boundary conditions for transported probability density function methods in transported probability density function pdf methods the pdf transport equations are most often solved with realization based techniques using particles in this paper a consistent treatment of particle in and outflow boundary conditions for transported pdf methods is devised it is shown that the presented approach is simple and most important consistent with the underlying eulerian pdf transport equation this is not the case for other boundary condition implementations discussed in the literature especially if the fluctuating particle velocities are high compared with the averaged ones

hybrid continuum molecular simulations of transient gas flows with rarefaction an adaptively coupled continuum molecular approach for compressible viscous flows in transient calculations is presented the continuum domain is described by the unsteady compressible navier stokes equations and the molecular domain is solved by direct simulation monte carlo a strategy is described to extend the overlapped schwarz method with dirichlet dirichlet boundary condition coupling procedure to transient simulations the method has been successfully validated against full direct simulation monte carlo results for transient simulations of a one dimensional shock tube and a two dimensional pressure driven slit flow a sensitivity analysis showed that the used overlapped schwarz coupling method with dirichlet dirichlet boundary conditions is only weakly sensitive to various parameters e g the exact position of the continuum molecular interface the size of the overlap region and the scatter in the molecular solution which is a clear advantage over the more commonly used flux based coupling technique another advantage of the chosen coupling method is that the continuum and molecular time steps can be decoupled and that in general a coupling time step can be used that is much larger than the molecular time step the study performed also highlighted a limit of the method it was found to be necessary to keep the courant number based on the coupling time step the cell size in the continuum region and the molecular most probable velocity below 1 to avoid instabilities

object detection with dog scale space a multiple kernel learning approach difference of gaussians dog scale space for an image is a significant way to generate features for object detection and classification while applying dog scale space features for object detection classification we face two inevitable issues dealing with high dimensional data and selecting weighting of proper scales the scale selection process is mostly ad hoc to present in this paper we propose a multiple kernel learning mkl method for both dog scale selection weighting and dealing with high dimensional scale space data we design a novel shift invariant kernel function for dog scale space to select only the useful scales in the dog scale space a novel framework of mkl is also proposed we utilize a 1 norm support vector machine svm in the mkl optimization problem for sparse weighting of scales from dog scale space the optimized data dependent kernel accommodates only a few scales that are most discriminatory according to the large margin principle with a 2 norm svm this learned kernel is applied to a challenging detection problem in oil sand mining to detect large lumps in oil sand videos we tested our method on several challenging oil sand data sets our method yields encouraging results on these difficult to process images and compares favorably against other popular multiple kernel methods

taming chaotic systems with dithers abstract a simple approach is proposed to tame chaos by injecting high frequency signals commonly called dithers into the chaotic systems based on the relaxed method an appropriate dither is introduced to suppress chaotic motion if the frequency of dither is high enough the trajectory described by the dithered chaotic system and the trajectory of its corresponding mathematical model the relaxed system would be made as close as desired this fact enables us to get a rigorous prediction of the dithered chaotic system s behavior by obtaining the behavior of the relaxed system finally an example with simulations is given to illustrate the concepts discussed throughout this paper

importance sampling variance reduction for the fokker planck rarefied gas particle method the fokker planck approximation to the boltzmann equation solved numerically by stochastic particle schemes is used to provide estimates for rarefied gas flows this paper presents a variance reduction technique for a stochastic particle method that is able to greatly reduce the uncertainty of the estimated flow fields when the characteristic speed of the flow is small in comparison to the thermal velocity of the gas the method relies on importance sampling requiring minimal changes to the basic stochastic particle scheme we test the importance sampling scheme on a homogeneous relaxation planar couette flow and a lid driven cavity flow and find that our method is able to greatly reduce the noise of estimated quantities significantly we find that as the characteristic speed of the flow decreases the variance of the noisy estimators becomes independent of the characteristic speed

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

a hybrid algorithm for the joint pdf equation of turbulent reactive flows in this paper a new particle finite volume hybrid algorithm for the joint velocityfrequency composition pdf method for turbulent reactive flows is presented this method is a combination of a finite volume scheme and a particle method the finitevolume scheme is used to solve the reynolds averaged navier stokes equations and the particle method to solve the joint pdf transport equation the motivation is to reduce the bias and the statistical error and to have an algorithm which is more efficient than stand alone particle mesh methods therefore in the particle method we use the smoother mean densityh i and favre averaged velocity u fields computed

exact 1 norm support vector machines via unconstrained convex differentiable minimization support vector machines utilizing the 1 norm typically set up as linear programs mangasarian 2000 bradley and mangasarian 1998 are formulated here as a completely unconstrained minimization of a convex differentiable piecewise quadratic objective function in the dual space the objective function which has a lipschitz continuous gradient and contains only one additional finite parameter can be minimized by a generalized newton method and leads to an exact solution of the support vector machine problem the approach here is based on a formulation of a very general linear program as an unconstrained minimization problem and its application to support vector machine classification problems the present approach which generalizes both mangasarian 2004 and fung and mangasarian 2004 is also applied to nonlinear approximation where a minimal number of nonlinear kernel functions are utilized to approximate a function from a given number of function values

comparison of ell _ 1 norm svr and sparse coding algorithms for linear regression support vector regression svr is a popular function estimation technique based on vapnik s concept of support vector machine among many variants the ell 1 norm svr is known to be good at selecting useful features when the features are redundant sparse coding sc is a technique widely used in many areas and a number of efficient algorithms are available both ell 1 norm svr and sc can be used for linear regression in this brief the close connection between the ell 1 norm svr and sc is revealed and some typical algorithms are compared for linear regression the results show that the sc algorithms outperform the newton linear programming algorithm an efficient ell 1 norm svr algorithm in efficiency the algorithms are then used to design the radial basis function rbf neural networks experiments on some benchmark data sets demonstrate the high efficiency of the sc algorithms in particular one of the sc algorithms the orthogonal matching pursuit is two orders of magnitude faster than a well known rbf network designing algorithm the orthogonal least squares algorithm

knowledge based kernel approximation prior knowledge in the form of linear inequalities that need to be satisfied over multiple polyhedral sets is incorporated into a function approximation generated by a linear combination of linear or nonlinear kernels in addition the approximation needs to satisfy conventional conditions such as having given exact or inexact function values at certain points determining such an approximation leads to a linear programming formulation by using nonlinear kernels and mapping the prior polyhedral knowledge in the input space to one defined by the kernels the prior knowledge translates into nonlinear inequalities in the original input space through a number of computational examples including a real world breast cancer prognosis dataset it is shown that prior knowledge can significantly improve function approximation

fokker planck dsmc algorithm for simulations of rarefied gas flows a fokker planck based particle monte carlo algorithm was devised recently for simulations of rarefied gas flows by the authors 1 3 the main motivation behind the fokker planck fp model is computational efficiency which could be gained due to the fact that the resulting stochastic processes are continuous in velocity space this property of the model leads to simulations where the computational cost becomes independent of the knudsen number kn 3 however the fokker planck model which can be seen as a diffusion approximation of the boltzmann equation becomes less accurate as kn increases in this study we propose a hybrid fokker planck direct simulation monte carlo fp dsmc solution method which is applicable for the whole range of kn the objective of this algorithm is to retain the efficiency of the fp scheme at low kn kn 1 and to employ conventional dsmc at high kn kn 1 since the computational particles employed by the fp model represent the same data as in dsmc the coupling between the two methods is straightforward the new ingredient is a switching criterion which would ideally result in a hybrid scheme with the efficiency of the fp method and the accuracy of dsmc for the whole kn range here we adopt the number of collisions in a given computational cell and for a given time step size as a decision criterion in order to switch between the fp model and dsmc for assessment of the hybrid algorithm different test cases including flow impingement and flow expansion through a slit were studied both accuracy and efficiency of the model are shown to be excellent for the presented test cases

the lattice boltzmann equation a new tool for computational fluid dynamics abstract we present a series of applications which demonstrate that the lattice boltzmann equation is an adequate computational tool to address problems spanning a wide spectrum of fluid regimes ranging from laminar to fully turbulent flows in two and three dimensions

subsampled hessian newton methods for supervised learning newton methods can be applied in many supervised learning approaches however for large scale data the use of the whole hessian matrix can be time consuming recently subsampled newton methods have been proposed to reduce the computational time by using only a subset of data for calculating an approximation of the hessian matrix unfortunately we find that in some situations the running speed is worse than the standard newton method because cheaper but less accurate search directions are used in this work we propose some novel techniques to improve the existing subsampled hessian newton method the main idea is to solve a two dimensional subproblem per iteration to adjust the search direction to better minimize the second order approximation of the function value we prove the theoretical convergence of the proposed method experiments on logistic regression linear svm maximum entropy and deep networks indicate that our techniques significantly reduce the running time of the subsampled hessian newton method the resulting algorithm becomes a compelling alternative to the standard newton method for large scale data classification

a multiblock joint pdf finite volume hybrid algorithm for the computation of turbulent flows in complex geometries a new hybrid joint probability density function jpdf solution algorithm for turbulent flows in complex 3d geometries is presented the main focus is to demonstrate the applicability of jpdf methods for complex flows as observed in industrial applications all elements of the algorithm are explained in detail and extensive validation studies are presented a multiblock finite volume solver capable of handling globally unstructured locally structured grids was implemented together with a lagrangian particle method efficient and robust particle management and accurate schemes for estimation and interpolation of particle statistics have been developed for numerical efficiency particle sub time stepping and an implicit finite volume solver are applied a fast coupling strategy was developed together with a multigrid method that allows very fast convergence on refined grids comparison with an established jpdf code for a bluff body stabilized flow shows very good agreement furthermore robustness and consistency of the algorithm for turbulent flow simulations with complex geometry is demonstrated

parallel fokker planck dsmc algorithm for rarefied gas flow simulation in complex domains at all knudsen numbers a major challenge for the conventional direct simulation monte carlo dsmc technique lies in the fact that its computational cost becomes prohibitive in the near continuum regime where the knudsen number kn characterizing the degree of rarefaction becomes small in contrast the fokker planck fp based particle monte carlo scheme allows for computationally efficient simulations of rarefied gas flows in the low and intermediate kn regime the fokker planck collision operator instead of performing binary collisions employed by the dsmc method integrates continuous stochastic processes for the phase space evolution in time this allows for time step and grid cell sizes larger than the respective collisional scales required by dsmc dynamically switching between the fp and the dsmc collision operators in each computational cell is the basis of the combined fp dsmc method which has been proven successful in simulating flows covering the whole kn range until recently this algorithm had only been applied to two dimensional test cases in this contribution we present the first general purpose implementation of the combined fp dsmc method utilizing both shared and distributed memory parallelization this implementation provides the capability for simulations involving many particles and complex geometries by exploiting state of the art computer cluster technologies

parallel gradient distribution in unconstrained optimization a parallel version is proposed for a fundamental theorem of serial unconstrained optimization the parallel theorem allows each of k parallel processors to use simultaneously a different algorithm such as a descent newton quasi newton or conjugate gradient algorithm each processor can perform one or many steps of a serial algorithm on a portion of the gradient of the objective function assigned to it independently of the other processors eventually a synchronization step is performed which for differentiable convex functions consists of taking a strong convex combination of the k points found by the k processors a more general synchronization step applicable to convex as well as nonconvex functions consists of taking the best point found by the k processors or any point that is better the fundamental result that we establish is that any accumulation point of the parallel algorithm is stationary for the nonconvex case and is a global solution for the convex case computational testing on the thinking machines cm 5 multiprocessor indicates a speedup of the order of the number of processors employed

boundary conditions for regularized 13 moment equations for micro channel flows boundary conditions are the major obstacle in simulations based on advanced continuum models of rarefied and micro flows of gases in this paper we present a theory how to combine the regularized 13 moment equations derived from boltzmann s equation with boundary conditions obtained from maxwell s kinetic accommodation model while for the linear case these kinetic boundary conditions suffice we need additional conditions in the non linear case these are provided by the bulk solutions obtained after properly transforming the equations while keeping their asymptotic accuracy with respect to boltzmann s equation after finding a suitable set of boundary conditions and equations a numerical method for generic shear flow problems is formulated several test simulations demonstrate the stable and oscillation free performance of the new approach

a diffusive information preservation method for small knudsen number flows the direct simulation monte carlo dsmc method is a powerful particle based method for modeling gas flows it works well for relatively large knudsen kn numbers typically larger than 0 01 but quickly becomes computationally intensive as kn decreases due to its time step and cell size limitations an alternative approach was proposed to relax or remove these limitations based on replacing pairwise collisions with a stochastic model corresponding to the fokker planck equation j comput phys 229 1077 2010 j fluid mech 680 574 2011 similar to the dsmc method the downside of that approach suffers from computationally statistical noise to solve the problem a diffusion based information preservation d ip method has been developed the main idea is to track the motion of a simulated molecule from the diffusive standpoint and obtain the flow velocity and temperature through sampling and averaging the ip quantities to validate the idea and the corresponding model several benchmark problems with kn 10 3 10 4 have been investigated it is shown that the ip calculations are not only accurate but also efficient because they make possible using a time step and cell size over an order of magnitude larger than the mean collision time and mean free path respectively

knowledge based linear programming we introduce a class of linear programs with constraints in the form of implications such linear programs arise in support vector machine classification where in addition to explicit datasets to be classified prior knowledge such as expert experience in the form of logical implications are imposed on the classifier the overall problem can be viewed either as a semi infinite linear program or as a linear program with equilibrium constraints which in either case can be solved by an equivalent simple linear program under mild assumptions