efficient recursive algorithms for detection of abrupt changes in signals and control systems this paper addresses a number of open problems concerning the generalized likelihood ratio glr rules for online detection of faults and parameter changes in control systems it is shown that with an appropriate choice of the threshold and window size these glr rules are asymptotically optimal the rules are also extended to non likelihood statistics that are widely used in monitoring adaptive algorithms for system identification and control by establishing gaussian approximations to these statistics when the window size is chosen suitably recursive algorithms are developed for practical implementation of the procedure and importance sampling techniques are introduced for determining the threshold of the rule to satisfy prescribed bounds on the false alarm rate

a dependent data extension of wald s identity and its application to sequential test performance computation the development of an exponential transformation of measure not restricted to independent identically distributed data is presented for a class of semi markov processes known as markov additive ma process the ma measure transformation is applied to obtain a new form of the wald identity to demonstrate the utility of this extension to ma processes the application of sequential tests with markov data to error probability performance analysis is investigated in particular it is demonstrated how previous work dealing with robust error probability evaluation in the presence of uncertain serial dependency can be applied to sequential tests

an improved version of the augmented e constraint method augmecon2 for finding the exact pareto set in multi objective integer programming problems generation or a posteriori methods in multi objective mathematical programming momp is the most computationally demanding category among the momp approaches due to the dramatic increase in computational speed and the improvement of mathematical programming algorithms the generation methods become all the more attractive among today s decision makers in the current paper we present the generation method augmecon2 which is an improvement of our development augmecon although augmecon2 is a general purpose method we will demonstrate that augmecon2 is especially suitable for multi objective integer programming moip problems specifically augmecon2 is capable of producing the exact pareto set in moip problems by appropriately tuning its running parameters in this context we compare the previous and the new version in a series of new and old benchmarks found in the literature we also compare augmecon2 s performance in the generation of the exact pareto sets with established methods and algorithms based on specific moip problems knapsack set packing and on published results except from other mathematical programming methods augmecon2 is found to be competitive also with multi objective meta heuristics momh in producing adequate approximations of the pareto set in multi objective combinatorial optimization moco problems

arbitrarily varying channels with constrained inputs and states random coding theorems are proved for discrete memoryless arbitrarily varying channels avcs with constraints on the transmitted codewords and channel state sequences two types of constraints are considered peak i e required for each n length sequence almost surely and average over the message set or over an ensemble for peak constraints on the codewords and on the channel state sequences the avc is shown to have a strong random coding capacity if the codewords and or the channel state sequences are constrained in the average sense the avcs do not possess strong capacities only epsilon capacities are shown to exist

asymptotic optimality of invariant sequential probability ratio tests it is well known that wald s sprt for testing simple hypotheses based on i i d observations minimizes the expected sample size both under the null and under the alternative hypotheses among all tests with the same or smaller error probabilities and with finite expected sample sizes under the two hypotheses in this paper it is shown that this optimum property can be extended at least asymptotically as the error probabilities tend to 0 to invariant sprts like the sequential t test the savage sethuraman sequential rank order test etc in fact not only do these invariant sprts asymptotically minimize the expected sample size but they also asymptotically minimize all the moments of the sample size distribution among all invariant tests with the same or smaller error probabilities modifications of these invariant sprts to asymptotically minimize the moments of the sample size at an intermediate parameter are also considered

asymptotic normality of the maximum likelihood estimator for general hidden markov models hidden markov models hmms have during the last decade become a widespread tool for modeling sequences of dependent random variables inference for such models is usually based on the maximum likelihood estimator mle and consistency of the mle for general hmms was recently proved by leroux in this paper we show that under mild conditions the mle is also asymptotically normal and prove that the observed information matrix is a consistent estimator of the fisher information

exact algorithm for bi objective 0 1 knapsack problem in this paper we propose a new procedure based on linear mathematical programming formulation to find all efficient solutions of bi objective 0 1 knapsack problem we solve a limited number of single objective problems to find efficient solutions of bi objective 0 1 knapsack problem we show that the proposed method is able to generate efficient solutions with less constraints and variables than the general linear program found in the literature an example is presented to illustrate the steps of the procedure

generation of the exact pareto set in multi objective traveling salesman and set covering problems abstract the calculation of the exact set in multi objective combinatorial optimization moco problems is one of the most computationally demanding tasks as most of the problems are np hard in the present work we use augmecon2 a multi objective mathematical programming momp method which is capable of generating the exact pareto set in multi objective integer programming moip problems for producing all the pareto optimal solutions in two popular moco problems the multi objective traveling salesman problem motsp and the multi objective set covering problem moscp the computational experiment is confined to two objective problems that are found in the literature the performance of the algorithm is slightly better to what is already found from previous works and it goes one step further generating the exact pareto set to till now unsolved problems the results are provided in a dedicated site and can be useful for benchmarking with other momp methods or even multi objective meta heuristics momh that can check the performance of their approximate solution against the exact solution in motsp and moscp problems

information bounds and quick detection of parameter changes in stochastic systems by using information theoretic bounds and sequential hypothesis testing theory this paper provides a new approach to optimal detection of abrupt changes in stochastic systems this approach not only generalizes previous work in the literature on optimal detection far beyond the relatively simple models treated but also suggests alternative performance criteria which are more tractable and more appropriate for general stochastic systems in addition it leads to detection rules which have manageable computational complexity for on line implementation and yet are nearly optimal under the different performance criteria considered

decision theoretic optimality of the cusum procedure suppose x 1 x 2 are independent random variables such that for some unknown each of x 1 x 1 is distributed according to f 0 while x x 1 are all distributed according to f 1 we prove a result of moustakides that claims that the cusum procedures are optimal in the sense of lorden we do that by proving that the procedures are bayes for some stochastic mechanism of generating

a note on the computation of the cp rank abstract the purpose of this note is to address the computational question of determining whether or not a square nonnegative matrix over the reals is completely positive and finding its cp rank when it is we show that these questions can be resolved by finite algorithms and we provide non polynomial complexity bounds on the number of arithmetic boolean operations that these algorithms require we state several open questions including the existence of polynomial algorithms to resolve the above problems and availability of algorithms for addressing complete positivity over the rationals and over 0 1 matrices

universal classification for hidden markov models binary hypotheses testing using empirically observed statistics is studied in the neyman pearson formulation for the hidden markov model hmm an asymptotically optimal decision rule is proposed and compared to the generalized likelihood ratio test glrt which has been shown in earlier studies to be asymptotically optimal for simpler parametric families the proof of the main theorem is provided the result can be applied to several types of hmms commonly used in speech recognition and communication applications several applications are demonstrated

optimal stopping times for detecting changes in distributions it is shown that page s stopping time is optimal for the detection of changes in distributions in a well defined sense this work is a generalization of an existing result where it was shown that page s stopping time is optimal asymptotically

a factorization method for completely positive matrices abstract a matrix a is called completely positive if there exists an entrywise nonnegative matrix b such that a b b t these matrices play a major role in combinatorial and quadratic optimization in this paper we study the problem of finding a nonnegative factorization b b t of a given completely positive matrix a we formulate this factorization problem as a nonconvex feasibility problem and develop a solution method based on alternating projections a local convergence result can be shown for this algorithm we also provide a heuristic extension which improves the numerical performance of the algorithm extensive numerical tests show that the factorization method is very fast in most of the test instances and performs better than existing algorithms

algorithmic copositivity detection by simplicial partition we present new criteria for copositivity of a matrix i e conditions which ensure that the quadratic form induced by the matrix is nonnegative over the nonnegative orthant these criteria arise from the representation of the quadratic form in barycentric coordinates with respect to the standard simplex and simplicial partitions thereof we show that as the partition gets finer and finer the conditions eventually capture all strictly copositive matrices we propose an algorithmic implementation which considers several numerical aspects as an application we present results on the maximum clique problem we also briefly discuss extensions of our approach to copositivity with respect to arbitrary polyhedral cones

rational factorizations of completely positive matrices abstract in this note it is proved that every rational matrix which lies in the interior of the cone of completely positive matrices also has a rational cp factorization

a simplex algorithm for rational cp factorization in this paper we provide an algorithm similar to the simplex algorithm which determines a rational cp factorization of a given matrix whenever the matrix allows such a factorization this algorithm can be used to show that every integral completely positive 2 times 2 matrix has an integral cp factorization

a method for generating all efficient solutions of 0 1 multi objective linear programming problem in this paper a method is proposed to find all efficient solutions of 0 1 multi objective linear programming molp problem these solutions are specified without generating all feasible solutions corresponding to a feasible solution of 0 1 molp problem a vector is constructed which its components are the values of objective functions the method consists of a one stage algorithm in each iteration of this algorithm for obtaining some efficient solutions of 0 1 molp problem a 0 1 single objective linear programming problem is solved

universal decoding for finite state channels universal decoding procedures for finite state channels are discussed although the channel statistics are not known universal decoding can achieve an error probability with an error exponent that for large enough block length or constraint length in case of convolutional codes is equal to the random coding error exponent associated with the optimal maximum likelihood decoding procedure for the given channel the same approach is applied to sequential decoding yielding a universal sequential decoding procedure with a cutoff rate and an error exponent that are equal to those achieved by the classical sequential decoding procedure

factorization and cutting planes for completely positive matrices by copositive projection we consider the problem of projecting a matrix onto the cones of copositive and completely positive matrices as this can not be done directly we use polyhedral approximations of the cones with the help of these projections we obtain a technique to compute factorizations of completely positive matrices we also describe a method to determine a cutting plane which cuts off an arbitrary matrix from the completely positive or copositive cone

markovian structures in biological sequence alignments abstract the alignment of multiple homologous biopolymer sequences is crucial in research on protein modeling and engineering molecular evolution and prediction in terms of both gene function and gene product structure in this article we provide a coherent view of the two recent models used for multiple sequence alignment the hidden markov model hmm and the block based motif model to develop a set of new algorithms that have both the sensitivity of the block based model and the flexibility of the hmm in particular we decompose the standard hmm into two components the insertion component which is captured by the so called propagation model and the deletion component which is described by a deletion vector such a decomposition serves as a basis for rational compromise between biological specificity and model flexibility furthermore we introduce a bayesian model selection criterion that in combination with the propagation model genetic algorithm and other computational aspects forms the cor

a generalized change detection problem the purpose of this paper is to give a new statistical approach to the change diagnosis detection isolation problem the change detection problem has received extensive research attention however the change isolation problem has for the most part been ignored we consider a stochastic dynamical system with abrupt changes and investigate the multiple hypotheses extension of lorden s 1971 results we introduce a joint criterion of optimality for the detection isolation problem and then design a change detection isolation algorithm we also investigate the statistical properties of this algorithm we prove a lower bound for the criterion in a class of sequential change detection isolation algorithms it is shown that the proposed algorithm is asymptotically optimal in this class the theoretical results are applied to the case of additive changes in linear stochastic models

specification testing in markov switching time series models this paper develops a series of specification tests of markov switching time series models tests for omitted autocorrelation omitted arch misspecification of the markovian dynamics and omitted explanatory variables are proposed all of the tests can be constructed as a natural byproduct of the routine used to calculate the smoothed probability that a given observation came from a particular regime and do not require estimation of additional parameters the paper performs monte carlo analysis of the tests and briefly illustrates their use with an empirical application

solving the bi objective multi dimensional knapsack problem exploiting the concept of core this paper deals with the bi objective multi dimensional knapsack problem we propose the adaptation of the core concept that is effectively used in single objective multi dimensional knapsack problems the main idea of the core concept is based on the divide and conquer principle namely instead of solving one problem with n variables we solve several sub problems with a fraction of n variables core variables the quality of the obtained solution can be adjusted according to the size of the core and there is always a trade off between the solution time and the quality of solution in the specific study we define the core problem for the multi objective multi dimensional knapsack problem after defining the core we solve the bi objective integer programming that comprises only the core variables using the multicriteria branch and bound algorithm that can generate the complete pareto set in small and medium size multi objective integer programming problems a small example is used to illustrate the method while computational and economy issues are also discussed computational experiments are also presented using available or appropriately modified benchmarks in order to examine the quality of pareto set approximation with respect to the solution time extensions to the general multi objective case as well as to the computation of the exact solution are also mentioned

nonparametric sequential change point detection by a vertically trimmed box method this paper examines a new method for sequential detection of a sudden and unobservable change in a sequence of independent observations with completely unspecified distribution functions a nonparametric detection rule is proposed which relies on the concept of a moving vertically trimmed box as such it will be coined as the vertical box control chart v box chart its implementation requires merely to count the number of data points which fall into the box attached to the last available observation no a priori knowledge of data distributions is required and proper tuning of the box size provides a quick detection technique this is supported by establishing statistical properties of the method which explain the role of the tuning parameters used in the v box chart these theoretical results are verified by simulation studies which indicate that the v box chart may provide quick detection with zero delay for jumps of moderate sizes its averaged run length to detection is more favorable than the one for the classical ewma method by comparison with the classical shewhart chart which was optimized for normal errors our method provides comparable or better performance

some np complete problems in quadratic and nonlinear programming in continuous variable smooth nonconvex nonlinear programming we analyze the complexity of checking whether a a given feasible solution is not a local minimum and b the objective function is not bounded below on the set of feasible solutions we construct a special class of indefinite quadratic programs with simple constraints and integer data and show that checking a or b on this class is np complete as a corollary we show that checking whether a given integer square matrix is not copositive is np complete

an algorithm for detecting a change in a stochastic process the problem of detecting a change from one given stationary and ergodic stochastic process to another such process is considered it is assumed that both stochastic processes are processes with memory and that they are mutually independent a sequential test is proposed and analyzed it is proved that the proposed test is asymptotically optimal in a mathematically precise sense

improved methods for calculating vectors of short length in a lattice including a complexity analysis the standard methods for calculating vectors of short length in a lattice use a reduction procedure followed by enumerating all vectors of z in a suitable box however it suffices to consider those x e z which lie in a suitable ellipsoid having a much smaller volume than the box we show in this paper that searching through that ellipsoid is in many cases much more efficient if combined with an appropriate reduction procedure our method allows to do computations in lattices of much higher dimensions several randomly constructed numerical examples illustrate the superiority of our new method over the known ones

sprt and cusum in hidden markov models in this paper we study the problems of sequential probability ratio tests for parameterized hidden markov models we investigate in some detail the performance of the tests and derive corrected brownian approximations for error probabilities and expected sample sizes asymptotic optimality of the sequential probability ratio test for testing simple hypotheses based on hidden markov chain data is established next we consider the cumulative sum cusum procedure for change point detection in this model based on the renewal property of the stopping rule cusum can be regarded as a repeated one sided sequential probability ratio test asymptotic optimality of the cusum procedure is proved in the sense of lorden 1971 motivated by the sequential analysis in hidden markov models wald s likelihood ratio identity and wald s equation for products of markov random matrices are also given we apply these results to several types of hidden markov models i i d hidden markov models switch gaussian regression and switch gaussian autoregression which are commonly used in digital communications speech recognition bioinformatics and economics

optimality of cusum rule approximations in change point detection problems application to nonlinear state space systems the well known cumulative sum cusum sequential rule for abrupt model change detection in stochastic dynamic systems relies on the knowledge of the probability density functions of the system output variables conditional on their past values and on the system functioning mode at each time step this paper shows how to build an asymptotically optimal detection rule under the common average run length arl constraint when these densities are not available but can be consistently estimated this is the case for nonlinear state space systems observed through output variables for such systems a new class of particle filters based on convolution kernels allows to get consistent estimates of the conditional densities leading to an optimal cusum like filter detection rule fdr

solving zero one multiple objective programs through implicit enumeration abstract research on algorithms designed to solve zero one multiple objective linear programs has been rather limited this paper presents a solution procedure for these problems based on the concept of implicit enumeration computational experience is reported and analyzed advantages disadvantages and extensions of the algorithm are discussed

order estimation and sequential universal data compression of a hidden markov source by the method of mixtures we consider first the estimation of the order i e the number of states of a discrete time finite alphabet stationary ergodic hidden markov source hms our estimator uses a description of the observed data in terms of a uniquely decodable code with respect to a mixture distribution obtained by suitably mixing a parametric family of distributions on the observation space this procedure avoids maximum likelihood calculations the order estimator is shown to be strongly consistent with the probability of underestimation decaying exponentially fast in the number n of observations while the probability of overestimation does not exceed cn sup 3 where c is a constant next we present a sequential algorithm for the uniquely decodable universal data compression of the hms which performs an on line estimation of source order followed by arithmetic coding this code asymptotically attains optimum average redundancy

using dea to obtain efficient solutions for multi objective 0 1 linear programs abstract this paper concerns the problem of a service oriented public sector entity to allocate limited resources to different activities while keeping conflicting objectives in mind the multi objective resource allocation problem mrap is to select activities to be performed the authors formulate the problem as a multi objective 0 1 linear problem the authors implement data envelopment analysis dea with the banker charnes and cooper s bcc model to measure the decision making unit s dmu efficiency in this study the production function is a mathematical statement relating the technological relationship between the objectives and resources of mrap each dmu presents a technological relationship i e dmu presents a relationship between resources and objectives this relationship gives information about the use of resources and satisfactoriness of objectives the inputs and outputs respectively outline resources and objectives the production possibility set represents feasible solutions for mrap moreover due to the multiple objectives of problems the method derives a solution set instead of an optimal solution in single objective ones this solution set a well known efficient solutions set forms the decision set of problems each dmu results from an alternative a combination of activities the production possibility set presents all the candidates of dmu the set of alternatives resulting in efficient dmus is efficient solutions of mrap the authors developed a two stage algorithm to generate and evaluate dmus the first stage generates a dmu with the maximum of the distance function the second stage is then used to evaluate the efficiency of the generated dmu