efficient algorithms for the inverse spanning tree problem the inverse spanning tree problem is to modify edge weights in a graph so that a given treet is a minimum spanning tree the objective is to minimize the cost of the deviation the cost of deviation is typically a convex function we propose algorithms here that are faster than all known algorithms for the problem our algorithm s run time for any convex inverse spanning tree problem iso nm log 2n for a graph onn nodes andm edges plus the time required to find the minima of then convex deviation functions this not only improves on the complexity of previously known algorithms for the general problem but also for algorithms devised for special cases for the case when the objective is weighted absolute deviation sokkalingam et al 1999 devised an algorithm with run timeo n2m log nc forc the maximum edge weight for the sum of absolute deviations our algorithm runs in timeo n2 logn matching the run time of a recent ahuja and orlin 2000 improvement for this case a new algorithm for bipartite matching in path graphs is reported here with complexity ofo n1 5 logn this leads to a second algorithm for the sum of absolute deviations running ino n1 5 logn logc steps

efficient solution generation for multiple objective linear programming based on extreme ray generation method abstract in this paper we consider solution generation method for multiple objective linear programming problems the set of efficient or pareto optimal solutions for the problems can be regarded as global information in multiple objective decision making situation in the past three decades as solution generation techniques various conventional algorithms based on simplex like approach with heavy computational burden were developed therefore the development of novel and useful directions in efficient solution generation method have been desired the purpose of this paper is to develop theoretical results and computational techniques of the efficient solution generation method based on extreme ray generation method that sequentially generates efficient points and rays by adding inequality constraints of the polyhedral feasible region

convex separable optimization is not much harder than linear optimization the polynomiality of nonlinear separable convex concave optimization problems on linear constraints with a matrix with small subdeterminants and the polynomiality of such integer problems provided the inteter linear version of such problems ins polynomial is proven this paper presents a general purpose algorithm for converting procedures that solves linear programming problems the conversion is polynomial for constraint matrices with polynomially bounded subdeterminants among the important corollaries of the algorithm is the extension of the polynomial solvability of integer linear programming problems with totally unimodular constraint matrix to integer separable convex programming an algorithm for finding a e accurate optimal continuous solution to the nonlinear problem that is polynomial in log 1 e and the input size and the largest subdeterminant of the constraint matrix is also presented these developments are based on proximity results between the continuous and integral optimal solutions for problems with any nonlinear separable convex objective function the practical feature of our algorithm is that is does not demand an explicit representation of the nonlinear function only a polynomial number of function evaluations on a prespecified grid

analysis of the objective space in multiple objective linear programming abstract a multiple objective linear program is defined by a matrix c consisting of the coefficients of the linear objectives and a convex polytope x defined by the linear constraints an analysis of the objective space y c x for this problem is presented a characterization between a face of y and the corresponding faces of x is obtained this result gives a necessary and sufficient condition for a face to be efficient the theory and examples demonstrate the collapsing simplification that occurs in mapping x to y these results form a basis for a new approach to analyzing multiple objective linear programs

finding all maximal efficient faces in multiobjective linear programming an algorithm for finding the whole efficient set of a multiobjective linear program is proposed from the set of efficient edges incident to a vertex a characterization of maximal efficient faces containing the vertex is given by means of the lexicographic selection rule of dantzig orden and wolfe a connectedness property of the set of dual optimal bases associated to a degenerate vertex is proved an application of this to the problem of enumerating all the efficient edges incident to a degenerate vertex is proposed our method is illustrated with numerical examples and comparisons with armand malivert s algorithm show that this new algorithm uses less computer time

learning boolean concepts in the presence of many irrelevant features abstract in many domains an appropriate inductive bias is the min features bias which prefers consistent hypotheses definable over as few features as possible this paper defines and studies this bias in boolean domains first it is shown that any learning algorithm implementing the min features bias requires ln l 2 p p ln n e training examples to guarantee pac learning a concept having p relevant features out of n available features this bound is only logarithmic in the number of irrelevant features for implementing the min features bias the paper presents five algorithms that identify a subset of features sufficient to construct a hypothesis consistent with the training examples focus 1 is a straightforward algorithm that returns a minimal and sufficient subset of features in quasi polynomial time focus 2 does the same task as focus 1 but is empirically shown to be substantially faster than focus 1 finally the simple greedy mutual information greedy and weighted greedy algorithms are three greedy heuristics that trade optimality for computational efficiency experimental studies are presented that compare these exact and approximate algorithms to two well known algorithms id3 and fringe in learning situations where many irrelevant features are present these experiments show that contrary to expectations the id3 and fringe algorithms do not implement good approximations of min features the sample complexity and generalization performance of the focus algorithms is substantially better than either id3 or fringe on learning problems where the min features bias is appropriate these experiments also show that among our three heuristics the weighted greedy algorithm provides an excellent approximation to the focus algorithms

local extremes runs strings and multiresolution the paper considers the problem of nonparametric regression with emphasis on controlling the number of local extremes two methods the run method and the taut string multiresolution method are introduced and analyzed on standard test beds it is shown that the number and locations of local extreme values are consistently estimated rates of convergence are proved for both methods the run method converges slowly but can withstand blocks as well as a high proportion of isolated outliers the rate of convergence of the taut string multiresolution method is almost optimal the method is extremely sensitive and can detect very low power peaks section 1 contains an introduction with special reference to the number of local extreme values the run method is described in section 2 and the taut string multiresolution method in section 3 low power peaks are considered in section 4 section 5 contains a comparison with other methods and section 6 a short conclusion the proofs are given in section 7 and the taut string algorithm is described in the appendix

an improved bl algorithm for genetic algorithm of the orthogonal packing of rectangles this paper proposes an improved bl algorithm for genetic algorithm of the orthogonal packing of rectangles some improvements on the fitness function of genetic algorithm for the orthogonal packing of rectangles are also suggested solutions of two numerical examples show the effectiveness of these improvements

a fast scaling algorithm for minimizing separable convex functions subject to chain constraints we consider the problem of minimizing j nc j x j subject to the following chain constraints x 1 x 2 x 3 x n where c j x j is a convex function of x j for each j n 1 2 n this problem is a generalization of theisotonic regression problems with complete order an important class of problems in regression analysis that has been examined extensively in the literature we refer to this problem as thegeneralized isotonic regression problem in this paper we focus on developing a fast scaling algorithm to obtain an integer solution of the generalized isotonic regression problem letu denote the difference between an upper bound on an optimal value of x n and a lower bound on an optimal value of x 1 under the assumption that the evaluation of any function c j x j takeso 1 time we show that the generalized isotonic regression problem can be solved ino n logu time this improves by a factor of n the previous best running time ofo n 2 logu to solve the same problem in addition when our algorithm is specialized to theisotonic median regression problem where c j x j c j x j a ji for specified values ofc js anda js the algorithm obtains a real valued optimal solution ino n logn time this time bound matches the best available time bound to solve the isotonic median regression problem but our algorithm uses simpler data structures and may be easier to implement

polynomial methods for separable convex optimization in unimodular linear spaces with applications we consider the problem of minimizing a separable convex objective function over the linear space given by a system mx 0 with m a totally unimodular matrix in particular this generalizes the usual minimum linear cost circulation and cocirculation problems in a network and the problems of determining the euclidean distance from a point to the perfect bipartite matching polytope and the feasible flows polyhedron we first show that the idea of minimum mean cycle canceling originally worked out for linear cost circulations by goldberg and tarjan j assoc comput mach 36 1989 pp 873 886 and extended to some other problems t r ervolina and s t mccormick discrete appl math 46 1993 pp 133 165 a frank and a v karzanov technical report rr 895 m laboratoire artemis imag universite joseph fourier grenoble france 1992 t ibaraki a v karzanov and h nagamochi private communication 1993 m hadjiat technical report groupe intelligence artificielle faculte des sciences de luminy marseille france 1994 can be generalized to give a combinatorial method with geometric convergence for our problem we also generalize the computationally more efficient cancel and tighten method we then consider objective functions that are piecewise linear pure and piecewise quadratic or piecewise mixed linear and quadratic and we show how both methods can be implemented to find exact solutions in polynomial time strongly polynomial in the piecewise linear case these implementations are then further specialized for finding circulations and cocirculations in a network we finish by showing how to extend our methods to find optimal integer solutions to linear spaces of larger fractionality and to the case when the objective functions are given by approximate oracles

bayesian variable selection in linear regression abstract this article is concerned with the selection of subsets of predictor variables in a linear regression model for the prediction of a dependent variable it is based on a bayesian approach intended to be as objective as possible a probability distribution is first assigned to the dependent variable through the specification of a family of prior distributions for the unknown parameters in the regression model the method is not fully bayesian however because the ultimate choice of prior distribution from this family is affected by the data it is assumed that the predictors represent distinct observables the corresponding regression coefficients are assigned independent prior distributions for each regression coefficient subject to deletion from the model the prior distribution is a mixture of a point mass at 0 and a diffuse uniform distribution elsewhere that is a spike and slab distribution the random error component is assigned a normal distribution with mean 0 and standard deviation

an efficient algorithm for image segmentation markov random fields and related problems problems of statistical inference involve the adjustment of sample observations so they fit some a priori rank requirements or order constraints in such problems the objective is to minimize the deviation cost function that depends on the distance between the observed value and the modify value in markov random field problems there is also a pairwise relationship between the objects the objective in markov random field problem is to minimize the sum of the deviation cost function and a penalty function that grows with the distance between the values of related pairs separation function we discuss markov random fields problems in the context of a representative application the image segmentation problem in this problem the goal is to modify color shades assigned to pixels of an image so that the penalty function consisting of one term due to the deviation from the initial color shade and a second term that penalizes differences in assigned values to neighboring pixels is minimized we present here an algorithm that solves the problem in polynomial time when the deviation function is convex and separation function is linear and in strongly polynomial time when the deviation cost function is linear quadratic or piecewise linear convex with few pieces where few means a number exponential in a polynomial function of the number of variables and constraints the complexity of the algorithm for a problem on n pixels or variables m adjacency relations or constraints and range of variable values colors u is o t n m n log u where t n m is the complexity of solving the minimum s t cut problem on a graph with n nodes and m arcs furthermore other algorithms are shown to solve the problem with convex deviation and convex separation in running time o mn log n log nu and the problem with nonconvex deviation and convex separation in running time o t nu mu the nonconvex separation problem is np hard even for fixed value of u for the family of problems with convex deviation functions and linear separation function the algorithm described here runs in polynomial time which is demonstrated to be fastest possible

improved local search algorithms for the rectangle packing problem with general spatial costs the rectangle packing problem with general spatial costs is to pack given rectangles without overlap in the plane so that the maximum cost of the rectangles is minimized this problem is very general and various types of packing problems and scheduling problems can be formulated in this form for this problem we have previously presented local search algorithms using a pair of permutations of rectangles to represent a solution in this paper we propose speed up techniques to evaluate solutions in various neighborhoods computational results for the rectangle packing problem and a real world scheduling problem exhibit good prospects of the proposed techniques

membership margin based feature selection for mixed type and high dimensional data definition of membership margin based objective function weighting the antecedents of fuzzy if then rules to improve classification performance evaluate the feature importance within membership margin framework feature selection algorithm for mixed type and high dimensional data the effectiveness of the approach is proved through an extensive experimental study the present paper describes a new feature weighting method based on a membership margin distinctive properties of the proposed method include its capability to process problems characterized by mixed type data quantitative qualitative and interval as well as a huge number of features the key idea is to map simultaneously all the features of different types into a common space the membership space once all features are represented in a homogeneous space a feature weighting task can be performed in unified way this weighting approach is integrated here within a fuzzy classifier through a fuzzy rule weighted concept in order to improve its performance each antecedent fuzzy set in the fuzzy if then rule is weighted to characterize the importance of each proposition and therefore its corresponding feature weight estimation process is based on membership margin maximization to estimate a fuzzy weight of each feature in the membership space experiments on low and high dimensional real world datasets demonstrate that the proposed approach can improve significantly the performance of the fuzzy rule based as well as other state of the art classifiers and can even outperform classical feature weighting approaches in particular we show that this approach can yield meaningful results on two real world applications for cancer prognosis and industrial process diagnosis

feature subset selection in large dimensionality domains searching for an optimal feature subset from a high dimensional feature space is known to be an np complete problem we present a hybrid algorithm saga for this task saga combines the ability to avoid being trapped in a local minimum of simulated annealing with the very high rate of convergence of the crossover operator of genetic algorithms the strong local search ability of greedy algorithms and the high computational efficiency of generalized regression neural networks we compare the performance over time of saga and well known algorithms on synthetic and real datasets the results show that saga outperforms existing algorithms

mining axiomatic fuzzy set association rules for classification problems abstract in this paper we propose a novel method to mine association rules for classification problems namely afsrc afs association rules for classification realized in the framework of the axiomatic fuzzy set afs theory this model provides a simple and efficient rule generation mechanism it can also retain meaningful rules for imbalanced classes by fuzzifying the concept of the class support of a rule in addition afsrc can handle different data types occurring simultaneously furthermore the new model can produce membership functions automatically by processing available data an extensive suite of experiments are reported which offer a comprehensive comparison of the performance of the method with the performance of some other methods available in the literature the experimental result shows that afsrc outperforms most of other methods when being quantified in terms of accuracy and interpretability afsrc forms a classifier with high accuracy and more interpretable rule base of smaller size while retaining a sound balance between these two characteristics

bayesian variable selection in generalized linear models using a combination of stochastic optimization methods abstract in this paper the usage of a stochastic optimization algorithm as a model search tool is proposed for the bayesian variable selection problem in generalized linear models combining aspects of three well known stochastic optimization algorithms namely simulated annealing genetic algorithm and tabu search a powerful model search algorithm is produced after choosing suitable priors the posterior model probability is used as a criterion function for the algorithm in cases when it is not analytically tractable laplace approximation is used the proposed algorithm is illustrated on normal linear and logistic regression models for simulated and real life examples and it is shown that with a very low computational cost it achieves improved performance when compared with popular mcmc algorithms such as the mcmc model composition as well as with vanilla versions of simulated annealing genetic algorithm and tabu search

regression and time series model selection in small samples summary a bias correction to the akaike information criterion aic is derived for regression and autoregressive time series models the correction is of particular use when the sample size is small or when the number of fitted parameters is a moderate to large fraction of the sample size the corrected method called aicc is asymptotically efficient if the true model is infinite dimensional furthermore when the true model is of finite dimension aicc is found to provide better model order choices than any other asymptotically efficient method applications to nonstationary autoregressive and mixed autoregressive moving average time series models are also discussed

optimization over the efficient set of a parametric multiple objective linear programming problem abstract in this paper we consider the problem p for optimizing a function over the efficient set of a multiple objective linear programming molp problem with parameters in the right hand side vector three solution algorithms in a most general case of problem p and improvements of them in some special cases are presented in the case when the objective function of problem p is linear it can be solved based on t 2 linear programming problems with mixed one zero integer variables if the parametric set is finite and based on t 2 linear programming problems if the right hand side vector of the molp problem is a linear function of the parameters and the parametric set is a polyhedron where t 2 is in general the number of maximal efficient faces of the molp problem corresponding to a value of the parameters a numerical example is given to illustrate the working of the algorithms

local search algorithms for the rectangle packing problem with general spatial costs we propose local search algorithms for the rectangle packing problem to minimize a general spatial cost associated with the locations of rectangles the problem is to pack given rectangles without overlap in the plane so that the maximum cost of the rectangles is minimized each rectangle has a set of modes where each mode specifies the width and height of the rectangle and its spatial cost function the spatial costs are general piecewise linear functions which can be non convex and discontinuous various types of packing problems and scheduling problems can be formulated in this form to represent a solution of this problem a pair of permutations of n rectangles is used to determine their horizontal and vertical partial orders respectively we show that under the constraint specified by such a pair of permutations optimal locations of the rectangles can be efficiently determined by using dynamic programming the search for finding good pairs of permutations is conducted by local search and metaheuristic algorithms we report computational results on various implementations using different neighborhoods and compare their performance we also compare our algorithms with other existing heuristic algorithms for the rectangle packing problem and scheduling problem these computational results exhibit good prospects of the proposed algorithms

a 98 effective lot sizing rule for a multi product multi stage production inventory system we study a multi product multi stage production inventory system in continuous time in process and finished goods are referred to as products and inventories of a single item held at different locations are treated as different products external demand can occur for any or all of the products at a constant product dependent rate when an order is placed for a product it is instantly delivered the set up cost is a function of the set of products being ordered at that point in time the goal is to schedule orders for each of the products over an infinite horizon so as to minimize the long run average cost we define a new class of policies in which each product uses a stationary interval of time between successive orders and the ratio of the order intervals of any two products is an integer power of two we show that there is always a policy in this class that is within 2 of optimal this extends a similar result for the one warehouse multi retailer system for assembly systems we show how to compute such a policy in 0n log n time for general systems a sequence of max flow min cut computations is required the algorithm is efficient for very large systems

a crossvalidatory aic for hard wavelet thresholding in spatially adaptive function estimation summary we consider the selection of a hard wavelet threshold for recovery of a signal embedded in additive gaussian white noise this is closely related to the problem of selection of a subset model in orthogonal normal linear regression we start with a discussion of donoho johnstone s 1994 universal method next we give a computationally efficient algorithm for implementing a crossvalidatory method proposed by nason 1996 then we propose and develop theory in support of a crossvalidatory version of aic which like universal thresholding and nason s method can be implemented in o n log n operations where n is the sample size a simulation study reveals that both of the crossvalidatory methods can outperform universal hard thresholding

exact algorithms for the two dimensional strip packing problem with and without rotations we propose exact algorithms for the two dimensional strip packing problem 2sp with and without 90 rotations we first focus on the perfect packing problem pp which is a special case of 2sp wherein all given rectangles are required to be packed without wasted space and design branch and bound algorithms introducing several branching rules and bounding operations a combination of these rules yields an algorithm that is especially efficient for feasible instances of pp we then propose several methods of applying the pp algorithms to 2sp our algorithms succeed in efficiently solving benchmark instances of pp with up to 500 rectangles and those of 2sp with up to 200 rectangles they are often faster than existing exact algorithms specially tailored for problems without rotations

calibration and empirical bayes variable selection for the problem of variable selection for the normal linear model selection criteria such as aic c p bic and ric have fixed dimensionality penalties such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical bayes formulation based on this calibration we propose empirical bayes selection criteria that use hyperparameter estimates instead of fixed choices for obtaining these estimates both marginal and conditional maximum likelihood methods are considered as opposed to traditional fixed penalty criteria these empirical bayes criteria have dimensionality penalties that depend on the data their performance is seen to approximate adaptively the performance of the best fixed penalty criterion across a variety of orthogonal and nonorthogonal set ups including wavelet regression empirical bayes shrinkage estimators of the selected coefficients are also proposed

a combined constraint space objective space approach for determining high dimensional maximal efficient faces of multiple objective linear programs abstract characterizations for efficient faces and certain maximal efficient faces of the objective set y of a linear k objective minimization problem are presented these characterizations are used to develop an algorithm for etermining high dimensional maximal efficient faces of y the algorithm requires as input an irredundant system of linear inequalities representing the efficiency equivalent polyhedron y y r k a procedure for obtaining such a representation for y has previously appeared in the literature and is included herein in order to make the paper self contained this latter procedure requires in part the generation of the efficient extreme points and efficient extreme rays of the constraint polyhedron hence the overall method proposed herein can be viewed as a combined constraint space objective space algorithm the algorithm is complete for problems with 2 and 3 objectives an illustrative numerical example is included

mixed integer second order cone programming formulations for variable selection in linear regression abstract this study concerns a method of selecting the best subset of explanatory variables in a multiple linear regression model goodness of fit measures for example adjusted r2 aic and bic are generally used to evaluate a subset regression model although variable selection with regard to these measures is usually performed with a stepwise regression method it does not always provide the best subset of explanatory variables in this paper we propose mixed integer second order cone programming formulations for selecting the best subset of variables with respect to adjusted r2 aic and bic computational experiments show that in terms of these measures the proposed formulations yield better solutions than those provided by common stepwise regression methods

using simulated annealing to optimize the feature selection problem in marketing applications the feature selection also specification problem is concerned with finding the most influential subset of predictors in predictive modeling from a much larger set of potential predictors that can contain hundreds of predictors the problem belongs to the realm of combinatorial optimization where the objective is to find the subset of variables that optimize the value of some goodness of fit function due to the dimensionality of the problem the feature selection problem belongs to the group of np hard problems most of the available predictors are noisy or redundant and add very little if any to the prediction power of the model using all the predictors in the model often results in strong over fitting and very poor predictions constructing a prediction model by checking out all possible subsets is impractical due to computational volume looking on the contribution of each predictor separately is not accurate because it ignores the inter correlations between predictors as a result no analytic solution is available for the feature selection problem requiring that one resorts to heuristics in this paper we employ the simulated annealing sa approach which is one of the leading stochastic search methods for specifying a large scale linear regression model the sa results are compared to the results of the more common stepwise regression swr approach for model specification the models are applied on realistic data sets in database marketing we also use simulated data sets to investigate what data characteristics make the swr approach equivalent to the supposedly more superior sa approach

the risk inflation criterion for multiple regression a new criterion is proposed for the evaluation of variable selection procedures in multiple regression this criterion which we call the risk inflation is based on an adjustment to the risk essentially the risk inflation is the maximum increase in risk due to selecting rather than knowing the correct predictors a new variable selection procedure is obtained which in the case of orthogonal predictors substantially improves on aic c p and bic and is close to optimal in contrast to aic c p and bic which use dimensionality penalties of 2 2 and log n respectively this new procedure uses a penalty 2 log p where p is the number of available predictors for the case of nonorthogonal predictors bounds for the optimal penalty are obtained

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

income prediction in the agrarian sector using product unit neural networks abstract european union financial subsidies in the agrarian sector are directly related to maintaining a sustainable farm income so its determination using for example the farm gross margin is a basic element in agrarian programs for sustainable development using this tool it is possible the identification of the agrarian structures that need financial support and to what extent it is needed however the process of farm gross margin determination is complicated and expensive because it is necessary to find the value of all the inputs consumed and outputs produced considering the circumstances mentioned the objectives of this research were to 1 select a representative and reduced set of easy to collect descriptive variables to estimate the gross margin of a group of olive tree farms in andalusia 2 investigate if artificial neural network models ann with two different types of basis functions sigmoidal and product units could effectively predict the gross margin of olive tree farms 3 compare the effectiveness of multiple linear quadratic and robust regression models versus ann and 4 validate the best mathematical model obtained for gross margin prediction by analysing realistic farm and farmer scenarios results from ann models specially the product unit ones have provided the most accurate gross margin predictions

determining maximal efficient faces in multiobjective linear programming problem finding an efficient or weakly efficient solution in a multiobjective linear programming molp problem is not a difficult task the difficulty lies in finding all these solutions and representing their structures since there are many convenient approaches that obtain all of the weakly efficient extreme points and weakly efficient extreme rays in an molp this paper develops an algorithm which effectively finds all of the weakly efficient maximal faces in an molp using all of the weakly efficient extreme points and extreme rays the proposed algorithm avoids the degeneration problem which is the major problem of the most of previous algorithms and gives an explicit structure for maximal efficient weak efficient faces consequently it gives a convenient representation of efficient weak efficient set using maximal efficient weak efficient faces the proposed algorithm is based on two facts firstly the efficiency and weak efficiency property of a face is determined using a relative interior point of it secondly the relative interior point is achieved using some affine independent points indeed the affine independent property enable us to obtain an efficient relative interior point rapidly

application of global optimization methods to model and feature selection many data mining applications involve the task of building a model for predictive classification the goal of this model is to classify data instances into classes or categories of the same type the use of variables not related to the classes can reduce the accuracy and reliability of classification or prediction model superfluous variables can also increase the costs of building a model particularly on large datasets the feature selection and hyper parameters optimization problem can be solved by either an exhaustive search over all parameter values or an optimization procedure that explores only a finite subset of the possible values the objective of this research is to simultaneously optimize the hyper parameters and feature subset without degrading the generalization performances of the induction algorithm we present a global optimization approach based on the use of cross entropy method to solve this kind of problem

subset selection in multiple linear regression models a hybrid of genetic and simulated annealing algorithms the question of variable selection in a multiple linear regression model is a major open research topic in statistics the subset selection problem in multiple linear regression deals with the selection of a minimal subset of input variables without loss of explanatory power in this paper we adapt the genetic and simulated annealing algorithms for variable selection in multiple linear regression the performance of this hybrid heuristic method is compared to those obtained by forward selection backward elimination and classical genetic algorithm search a comparative analysis on the literature data sets and simulation data shows that our hybrid heuristic method may suggest efficient alternative to traditional subset selection methods for the variable selection problem in multiple linear regression models

solving the convex cost integer dual network flow problem in this paper 1 we consider an integer convex optimization problem where the objective function is the sum of separable convex functions that is of the form i j eq f ij w ij iep b i i the constraints are similar to those arising in the dual of a minimum cost flow problem that is of the form i j w ij i j q with lower and upper bounds on variables letn p m q andu be the largest magnitude in the lower and upper bounds of variables we call this problemthe convex cost integer dual network flow problem in this paper we describe several applications of the convex cost integer dual network flow problem arising in a dial a ride transit problem inverse spanning tree problem project management and regression analysis we develop network flow based algorithms to solve the convex cost integer dual network flow problem we show that using the lagrangian relaxation technique the convex cost integer dual network flow problem can be transformed into a convex cost primal network flow problem where each cost function is a piecewise linear convex function with integer slopes its special structure allows the convex cost primal network flow problem to be solved ino nmlog n2 m log nu time this time bound is the same time needed to solve the minimum cost flow problem using the cost scaling algorithm and is also is best available time bound to solve the convex cost integer dual network flow problem

constructing efficient solutions structure of multiobjective linear programming it is not a difficult task to find a weak pareto or pareto solution in a multiobjective linear programming molp problem the difficulty lies in finding all these solutions and representing their structure this paper develops an algorithm for solving this problem we investigate the solutions and their relationships in the objective space the algorithm determines finite number of weights each of which corresponds to a weighted sum problems by solving these problems we further obtain all weak pareto and pareto solutions of the molp and their structure in the constraint space the algorithm avoids the degeneration problem which is a major hurdle of previous works and presents an easy and clear solution structure

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

multicriteria variable selection for classification of production batches abstract in many industrial processes hundreds of noisy and correlated process variables are collected for monitoring and control purposes the goal is often to correctly classify production batches into classes such as good or failed based on the process variables we propose a method for selecting the best process variables for classification of process batches using multiple criteria including classification performance measures i e sensitivity and specificity and the measurement cost the method applies partial least squares pls regression on the training set to derive an importance index for each variable then an iterative classification elimination procedure using k nearest neighbor is carried out finally pareto analysis is used to select the best set of variables and avoid excessive retention of variables the method proposed here consistently selects process variables important for classification regardless of the batches included in the training data further we demonstrate the advantages of the proposed method using six industrial datasets

variable selection via gibbs sampling abstract a crucial problem in building a multiple regression model is the selection of predictors to include the main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets this procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices in this framework the promising subsets of predictors can be identified as those with higher posterior probability the computational burden is then alleviated by using the gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices those subsets with higher probability the promising ones can then be identified by their more frequent appearance in the gibbs sample

the maximal descriptor index set for a face of a convex polyhedral set and some applications abstract a face of a convex polyhedral set can be investigated on the basis of only the maximal descriptor index set for it this helps us reduce difficulties and computational efforts in solving many problems in this paper we consider a multiple objective linear programming molp problem in which the constraint polyhedron may have no extreme points and the variables may be negative first we deal with determining the maximal descriptor index set for an arbitrary face of the constraint polyhedron then we apply the maximal descriptor index sets for faces to compare descriptor sets for the faces and find all maximal efficient faces for the molp problem the methods known so far compare descriptor sets at large computational cost in terms of computing times and storage because they require finding all extreme points and directions of them known top down search methods have been devised to find the efficient set of the molp problem but they may fail to find all maximal efficient faces and the computational cost is considerable we propose a new method to compare descriptor sets and a new top down search method which finds all maximal efficient faces and requires less computational cost a numerical example is given to illustrate how the method works

flexible empirical bayes estimation for wavelets summary wavelet shrinkage estimation is an increasingly popular method for signal denoising and compression although bayes estimators can provide excellent mean squared error mse properties the selection of an effective prior is a difficult task to address this problem we propose empirical bayes eb prior selection methods for various error distributions including the normal and the heavier tailed student t distributions under such eb prior distributions we obtain threshold shrinkage estimators based on model selection and multiple shrinkage estimators based on model averaging these eb estimators are seen to be computationally competitive with standard classical thresholding methods and to be robust to outliers in both the data and wavelet domains simulated and real examples are used to illustrate the flexibility and improved mse performance of these methods in a wide variety of settings

a discrete particle swarm optimization method for feature selection in binary classification problems abstract this paper investigates the feature subset selection problem for the binary classification problem using logistic regression model we developed a modified discrete particle swarm optimization pso algorithm for the feature subset selection problem this approach embodies an adaptive feature selection procedure which dynamically accounts for the relevance and dependence of the features included the feature subset we compare the proposed methodology with the tabu search and scatter search algorithms using publicly available datasets the results show that the proposed discrete pso algorithm is competitive in terms of both classification accuracy and computational performance

linear model selection by cross validation abstract we consider the problem of selecting a model having the best predictive ability among a class of linear models the popular leave one out cross validation method which is asymptotically equivalent to many other model selection methods such as the akaike information criterion aic the c p and the bootstrap is asymptotically inconsistent in the sense that the probability of selecting the model with the best predictive ability does not converge to 1 as the total number of observations n we show that the inconsistency of the leave one out cross validation can be rectified by using a leave n v out cross validation with n v the number of observations reserved for validation satisfying n v n 1 as n this is a somewhat shocking discovery because nv n 1 is totally opposite to the popular leave one out recipe in cross validation motivations justifications and discussions of some practical aspects of the use of the leave n v out cross validation method are provided and results

a variable selection method based on tabu search for logistic regression models a tabu search method is proposed and analysed for selecting variables that are subsequently used in logistic regression models the aim is to find from among a set of m variables a smaller subset which enables the efficient classification of cases reducing dimensionality has some very well known advantages that are summarized in literature the specific problem consists in finding for a small integer value of p a subset of size p of the original set of variables that yields the greatest percentage of hits in logistic regression the proposed tabu search method performs a deep search in the solution space that alternates between a basic phase that uses simple moves and a diversification phase to explore regions not previously visited testing shows that it obtains significantly better results than the stepwise backward or forward methods used by classic statistical packages some results of applying these methods are presented

convergence analysis of canonical genetic algorithms this paper analyzes the convergence properties of the canonical genetic algorithm cga with mutation crossover and proportional reproduction applied to static optimization problems it is proved by means of homogeneous finite markov chain analysis that a cga will never converge to the global optimum regardless of the initialization crossover operator and objective function but variants of cga s that always maintain the best solution in the population either before or after selection are shown to converge to the global optimum due to the irreducibility property of the underlying original nonconvergent cga these results are discussed with respect to the schema theorem