heuristics and reduction methods for multiple constraints 0 1 linear programming problems abstract this work extends the efficient results relative to the 0 1 knapsack problem to the multiple inequality constraints 0 1 linear programming problems the two crucial phases for the solving of this type of problems are presented i two linear expected time complexity greedy algorithms are proposed for the determination of a lower bound on the optimal value by using a cascade of surrogate relaxations of the original problem whose sizes are decreasing step by step a comparative study with the best known heuristic methods is reported it concerned the accuracy of the approximate solutions and the practical computational times ii this greedy algorithm is inserted in an efficient reduction framework variables and constraints are eliminated by the conjunction of tests applied to lagrangean and surrogate relaxations of the original problem a lot of computational results are summarized by considering test problems of the literature

statistical mechanics of the knapsack problem the knapsack problem is an np complete combinatorial optimization problem with inequality constraints using the replica method of statistical physics we study the space of its solutions for a large problem size it turns out that this problem is closely related to the theory of the binary perceptron

probabilistic analysis of the multidimensional knapsack problem we analyse the multi constraint zero one knapsack problem under the assumption that all coefficients are drawn from a uniform 0 1 distribution and there are m 01 constraints as the number of variables grows we show that results on the single constraint problem can be extended to this situation chiefly we generalise a result of lueker on the duality gap and a result of goldberg marchetti spaccamela on exact solvability in the latter case our methods differ markedly from those for the single constraint result

emulation of utility functions over a set of permutations sequencing reliability growth tasks abstractwe consider bayesian design of experiments problems in which we maximise the prior expectation of a utility function over a set of permutations for example when sequencing a number of tasks to perform when the number of tasks is large and the expected utility is expensive to compute it may be unreasonable or infeasible to evaluate the expected utility of all permutations we propose an approach to emulate the expected utility using a surrogate function based on a parametric probabilistic model for permutations the surrogate function is fitted by maximising the correlation with the expected utility over a set of training points we propose a suitable transformation of the expected utility to improve the fit we provide results linking the correlation between the two functions and the number of expected utility evaluations to undertake the approach is applied to the sequencing of reliability growth tasks in the development of hardware systems in which there is a large number of potential tasks

the tsallis entropy of natural information abstract estimating the information contained in natural data such as electroencephalography data is unusually difficult because the relationship between the physical data and the information that it encodes is unknown this unknown relationship is often called the encoding problem the present work provides a solution to this problem by deriving a method to estimate the tsallis entropy in natural data the method is based on two findings the first finding is that the physical instantiation of any information event that is the physical occurrence of a symbol of information must begin and end at a discontinuity or critical point maximum minimum or saddle point in the data the second finding is that in certain data types such as the encephalogram eeg the variance within of an eeg waveform event is directly proportional to its probability of occurrence these two outcomes yield two results the first is the easy binning of data into separate information events the second is the ability to estimate probabilities in two ways frequency counting and computing the variance within of an eeg waveform these results are used to derive a linear estimator of the tsallis entropy functional allowing it to be estimated without deducing the encoding this method for estimating the tsallis entropy is first used to estimate the information in simple signals the amount of information estimated is highly accurate the method is then applied to two problems in electroencephalography the first is distinguishing normal aging from very early alzheimer s disease mild cognitive impairment and the second is medication monitoring of alzheimer s disease treatment the former is done with an accuracy of 92 and the latter with an accuracy of 91 this detection accuracy is the highest published accuracy in the literature which suggests that this method for tsallis entropy estimation is both accurate and useful

neural networks for optimization problems with inequality constraints the knapsack problem a strategy for finding approximate solutions to discrete optimization problems with inequality constraints using mean field neural networks is presented the constraints x d 0 are encoded by x x terms in the energy function a careful treatment of the mean field approximation for the self coupling parts of the energy is crucial and results in an essentially parameter free algorithm this methodology is extensively tested on the knapsack problem of size up to 103 items the algorithm scales like nm for problems with n items and m constraints comparisons are made with an exact branch and bound algorithm when this is computationally possible n d 30 the quality of the neural network solutions consistently lies above 95 of the optimal ones at a significantly lower cpu expense for the larger problem sizes the algorithm is compared with simulated annealing and a modified linear programming approach for nonhomogeneous problems these produce good solutions whereas for the more difficult homogeneous problems the neural approach is a winner with respect to solution quality and or cpu time consumption the approach is of course also applicable to other problems of similar structure like set covering

an efficient tabu search approach for the 0 1 multidimensional knapsack problem abstract in this paper we describe a new approach to tabu search ts based on strategic oscillation and surrogate constraint information that provides a balance between intensification and diversification strategies new rules needed to control the oscillation process are given for the 0 1 multidimensional knapsack 0 1 mkp based on a portfolio of test problems from the literature our method obtains solutions whose quality is at least as good as the best solutions obtained by previous methods especially with large scale instances these encouraging results confirm the efficiency of the tunneling concept coupled with surrogate information when resource constraints are present

tsallis conditional mutual information in investigating long range correlation in symbol sequences abstract the presence of long range correlation lrc as opposed to markov chain of a finite order is of primary interest in the study of symbol sequences such as dna among others methods using information theory have been developed for markov chain order estimation in this work we consider the tsallis entropy and define the tsallis conditional mutual information tcmi for markov chain order estimation which is more suitable to identify large orders suggesting effectively lrc the tcmi of order m for a tsallis parameter q is computed for increasing m and a significance test is performed for each m until tcmi is found non significant randomization and parametric significance tests for tcmi are developed for the latter the null distribution of tcmi is approximated with a gamma distribution deriving analytic expressions for its parameters we assess the accuracy of order estimation with the two tests for tcmi and compare them to the respective tests using the shannon entropy extended simulations on markov chains of different orders and structures of the transition probability matrix show that shannon and tsallis tests with parameter q 2 have similar performance for small orders when the problem becomes more demanding for higher markov chain orders and lrcs tsallis tests for q 2 have better approximation to the correct order with the sequence length n finally the tsallis tests are favorably compared to shannon tests to real dna sequences

a revised bound improvement sequence algorithm abstract in this paper we present a generalization and a computational improvement of the bound improvement sequence algorithm the main computational burden of this algorithm consists in determining whether there exists a feasible point on the objective hyperplane when the algorithm encounters a fixed point by generalizing the algorithm such that the objective function and constraints are treated alike the number of fixed points that are required can be reduced the computational results that we report allow us to conclude that the number of fixed points can generally be reduced for loosely constrained problems for this class of problems the new algorithm appears to be more efficient than a standard mip code such as fmps

a hybrid dynamic programming branch and bound algorithm for the multiple choice knapsack problem abstract dynamic programming and branch and bound methodologies are combined to produce a hybrid algorithm for the multiple choice knapsack problem lagrangian duality is used in a computationally efficient manner to compute tight bounds on every active node in the search tree the use of lagrangian duality also enables the use of a reduction procedure to reduce the size of the problem for the enumeration phase computational experience with up to 200 multiple choice sets and 20000 zero one variables is reported the computational experience indicates that the resulting algorithm is faster than the best published algorithm and is simpler to code

probabilistic properties of the dual structure of the multidimensional knapsack problem and fast statistically efficient algorithms properties of several dual characteristics of the multidimensional knapsack problem such as the probability of existence of optimal and optimal feasible lagrange function generalized saddle points magnitude of relative duality gap etc are investigated for different probabilistic models sufficient conditions of good asymptotic behavior of the dual characteristics are given a fast statistically efficient approximate algorithm with linear running time complexity for problems with random coefficients is presented

human brain dynamics the analysis of eeg signals with tsallis information measure we undertake the study of human eeg signals by recourse to a wavelet based multiresolution analysis as adapted to an information measure scenario dierent information measures are employed it is shown that non extensive ones seem to be of particular usefulness their use opens up perspectives of building up automatic detection devices conjectures concerning general characteristics of focal epilepsy are formulated on the basis of a tsallis type of analysis c 1999 elsevier science b v all rights reserved

an exact algorithm for the reliability redundancy allocation problem abstract the redundancy allocation problem is the problem of finding an optimal allocation of redundant components subject to a set of resource constraints the problem studied in this paper refers to a series parallel system configuration and allows for component mixing we propose a new modeling solution approach in which the problem is transformed into a multiple choice knapsack problem and solved to optimality via a branch and cut algorithm the algorithm is tested on well known sets of benchmark instances all instances have been solved to optimality in milliseconds or very few seconds on a normal workstation

a note on approximation schemes for multidimensional knapsack problems polynomial and fully polynomial approximation algorithms for single dimensional knapsack problems have been extensively studied and a number of such algorithms constructed this note shows that the problem of finding a fully polynomial approximation algorithm for multidimensional knapsack problems is np hard

computational applications of nonextensive statistical mechanics computational applications of the nonextensive entropy s q and nonextensive statistical mechanics a current generalization of the boltzmann gibbs bg theory are briefly reviewed the corresponding bibliography is provided as well

surrogate dual multiplier search procedures in integer programming search procedures for optimal lagrange multipliers are highly developed and provide good bounds in branch and bound procedures that have led to the successful application of lagrangean duality in integer programming although the surrogate dual generally provides a better objective bound there has been little development of surrogate multiplier search procedures this paper develops and empirically analyzes several surrogate multiplier search procedures results indicate that the procedures can produce possibly superior bounds in an amount of time comparable to other techniques our discussion also highlights the similarity of the procedures to some well known lagrangean search techniques

on rates of convergence and asymptotic normality in the multiknapsack problem in meanti et al 1990 an almost sure asymptotic characterization has been derived for the optimal solution value as function of the knapsack capacities when the profit and requirement coefficients of items to be selected from are random variables in this paper we establish a rate of convergence for this process using results from the theory of empirical processes

solving zero one mixed integer programming problems using tabu search we describe a tabu search ts approach for solving general zero one mixed integer programming mip problems that exploits the extreme point property of zero one solutions specialized choice rules and aspiration criteria are identified for the problems expressed as functions of integer infeasibility measures and objective function values the first level ts mechanisms are then extended with advanced level strategies and learning we also look at probabilistic measures in this framework and examine how the learning tool target analysis ta can be applied to identify better control structures and decision rules computational results are reported on a portfolio of multiconstraint knapsack problems our approach is designed to solve thoroughly general 0 1 mip problems and thus contains no problem domain specific knowledge yet it obtains solutions for the multiconstraint knapsack problem whose quality rivals and in some cases surpasses the best solutions obtained by special purpose methods that have been created to exploit the special structure of these problems

reliability redundancy allocation an improved realization for nonconvex nonlinear programming problems the redundancy allocation problem which seeks to find the optimal allocation of redundancy that maximizes system reliability is one of the representative problems in reliability optimization the problem can be formulated as a nonconvex integer nonlinear programming problem this paper presents an efficient branch and bound approach to solve this problem where the considered system is coherent that is the objective and constraint functions have monotonic increasing properties the proposed method is based primarily on a search space elimination of disjoint sets in a solution space that does not require any relaxation of branched subproblems the main advantage of the proposed method is flexibility i e it does not rely on any assumptions of linearity separability single constraint or convexity which make the method adaptive to various applications numerical experiments demonstrate that the method is superior to the existing exact algorithms for redundancy allocation problems in terms of computation time

tabu search for the multilevel generalized assignment problem abstract the multilevel generalized assignment problem mgap differs from the classical gap in that agents can perform tasks at more than one efficiency level important manufacturing problems such as lot sizing can be formulated as mgaps however the large number of variables in the related 0 1 integer program makes the use of commercial optimization packages impractical in this paper we present a heuristic approach to the solution of the mgap which consists of a novel application of tabu search ts our ts method employs neighborhoods defined by ejection chains that produce moves of greater power without significantly increasing the computational effort

threshold accepting a general purpose optimization algorithm appearing superior to simulated annealing abstract a new general purpose algorithm for the solution of combinatorial optimization problems is presented the new threshold accepting method is even simpler structured than the wellknown simulated annealing approach the power of the new algorithm is demonstrated by computational results concerning the traveling salesman problem and the problem of the construction of error correcting codes moreover deterministic versions of the new heuristic turn out to perform nearly equally well consuming only a fraction of the computing time of the stochastic versions as an example the deterministic threshold accepting method yields very near to optimum tours for the famous 442 cities traveling salesman problem of grotschel within 1 to 2 s of cpu time

regression model selection a residual likelihood approach summary we obtain the residual information criterion ric a selection criterion based on the residual log likelihood for regression models including classical regression models box cox transformation models weighted regression models and regression models with autoregressive moving average errors we show that ric is a consistent criterion and that simulation studies for each of the four models indicate that ric provides better model order choices than the akaike information criterion corrected akaike information criterion final prediction error cp and r 2 except when the sample size is small and the signal to noise ratio is weak in this case none of the criteria performs well monte carlo results also show that ric is superior to the consistent bayesian information criterion bic when the signal to noise ratio is not weak and it is comparable with bic when the signal to noise ratio is weak and the sample size is large

a new algorithm for the 0 1 knapsack problem we present a new algorithm for the optimal solution of the 0 1 knapsack problem which is particularly effective for large size problems the algorithm is based on determination of an appropriate small subset of items and the solution of the corresponding core problem from this we derive a heuristic solution for the original problem which with high probability can be proved to be optimal the algorithm incorporates a new method of computation of upper bounds and efficient implementations of reduction procedures the corresponding fortran code is available we report computational experiments on small size and large size random problems comparing the proposed code with all those available in the literature

possible generalization of boltzmann gibbs statistics with the use of a quantity normally scaled in multifractals a generalized form is postulated for entropy namelys q k 1 i 1 w p i q q 1 whereq characterizes the generalization andp i are the probabilities associated withw microscopic configurations w the main properties associated with this entropy are established particularly those corresponding to the microcanonical and canonical ensembles the boltzmann gibbs statistics is recovered as theq 1 limit

joint estimation and testing for functional form and heteroskedasticity abstract this paper develops the maximum likelihood procedure to estimate the appropriate functional form in regression models with heteroskedastic errors the analysis can then enable us to separate out the influence of non linearity in an estimate of the transformation parameter from the influence of stabilizing the error variance illustrative examples have been used to show that estimation and tests for functional form and heteroskedasticity can and should be jointly considered

tsallis entropy and cortical dynamics the analysis of eeg signals abstract we undertake the study of eeg signals by recourse to a wavelet based multiresolution analysis as adapted to an information measure scenario different information measures are employed it is shown that non extensive ones seem to be of particular usefulness

a statistical analysis of the knapsack problem we investigate the dependence of the multi knapsack objective function on the knapsack capacities and on the number of capacity constraints p in the case when all n objects are assigned the same profit value and the weights are uniformly distributed over the unit interval a rigorous upper bound to the optimal profit is obtained employing the annealed approximation and then compared with the exact value obtained through the lagrangian relaxation method the analysis is restricted to the regime where n goes to infinity and p remains finite

model specification tests a simultaneous approach abstract in econometrics specification tests have been constructed to verify the validity of one specification at a time it is argued that most of these tests are not in general robust in the presence of other misspecifications so their application may result in misleading conclusions using the lagrange multiplier principle we develop efficient test procedures that are capable of testing a number of specifications simultaneously these tests will confirm the validity or invalidity of a general model requiring the estimates of the restricted model only through an extensive monte carlo experiment we study the performance of these tests and some commonly used one directional tests we also suggest a multiple comparison procedure to identify different sources of errors this we hope will lead to a better specification of econometric models

the growth of m constraint random knapsacks the author computes the asymptotic value of a particular m constraint n variable 0 1 random integer programming problem as n increases m remaining fixed this solves a problem of frieze and clarke 1984

an approximate algorithm for multidimensional zero one knapsack problems a new approximate algorithm for multidimensional zero one knapsack problems with all positive coefficients is presented the procedure is controlled by three parameters which affect the tradeoff between solution quality and computation time and whose values are set by the users for 48 test problems with 5 to 20 constraints and 6 to 500 variables the solution found was on the average within 0 34 of the optimum and the computation time was the shortest compared with three other well known heuristics

tsallis like information measures and the analysis of complex signals abstract a tsallis like information measure that elaborates upon those of the tsallis type is shown to be a rather powerful tool for the analysis of complex non stationary signals as exemplified by the henon map and eegs the ensuing results could be of importance concerning the automatic detection of epileptic spikes

the growth of multi constraint random knapsacks with various right hand sides of the constraints abstract the asymptotic value of the m constraint n variable 0 1 random knapsack problem with b j n the right hand side of the j th constraint m fixed n is computed previously known results have been generalized

new trends in exact algorithms for the 0 1 knapsack problem abstract while the 1980s were focused on the solution of large sized easy knapsack problems kps this decade has brought several new algorithms which are able to solve hard large sized instances we will give an overview of the recent techniques for solving hard kps with special emphasis on the addition of cardinality constraints dynamic programming and rudimentary divisibility computational results comparing all recent algorithms are presented

the multidimensional 0 1 knapsack problem an overview abstract the multidimensional 0 1 knapsack problem is one of the most well known integer programming problems and has received wide attention from the operational research community during the last four decades although recent advances have made possible the solution of medium size instances solving this np hard problem remains a very interesting challenge especially when the number of constraints increases this paper surveys the main results published in the literature the focus is on the theoretical properties as well as approximate or exact solutions of this special 0 1 program

wavelet analysis and nonlinear dynamics in a nonextensive setting abstract we undertake the study of signals originated in time dependent nonlinear systems by recourse to a wavelet based multiresolution analysis as adapted to a nonextensive tsallis scenario diverse applications are discussed it is shown that the tsallis environment provides one with more detailed information than the conventional shannon one

estimation of entropy and mutual information we present some new results on the nonparametric estimation of entropy and mutual information first we use an exact local expansion of the entropy function to prove almost sure consistency and central limit theorems for three of the most commonly used discretized information estimators the setup is related to grenander s method of sieves and places no assumptions on the underlying probability measure generating the data second we prove a converse to these consistency theorems demonstrating that a misapplication of the most common estimation techniques leads to an arbitrarily poor estimate of the true information even given unlimited data this inconsistency theorem leads to an analytical approximation of the bias valid in surprisingly small sample regimes and more accurate than the usual 1 n formula of miller and madow over a large region of parameter space the two most practical implications of these results are negative 1 information estimates in a certain data regime are likely contaminated by bias even if bias corrected estimators are used and 2 confidence intervals calculated by standard techniques drastically underestimate the error of the most common estimation methods finally we note a very useful connection between the bias of entropy estimators and a certain polynomial approximation problem by casting bias calculation problems in this approximation theory framework we obtain the best possible generalization of known asymptotic bias results more interesting this framework leads to an estimator with some nice properties the estimator comes equipped with rigorous bounds on the maximum error over all possible underlying probability distributions and this maximum error turns out to be surprisingly small we demonstrate the application of this new estimator on both real and simulated data