dimension reduction and coefficient estimation in multivariate linear regression summary we introduce a general formulation for dimension reduction and coefficient estimation in the multivariate linear model we argue that many of the existing methods that are commonly used in practice can be formulated in this framework and have various restrictions we continue to propose a new method that is more flexible and more generally applicable the method proposed can be formulated as a novel penalized least squares estimate the penalty that we employ is the coefficient matrix s ky fan norm such a penalty encourages the sparsity among singular values and at the same time gives shrinkage coefficient estimates and thus conducts dimension reduction and coefficient estimation simultaneously in the multivariate linear model we also propose a generalized cross validation type of criterion for the selection of the tuning parameter in the penalized least squares simulations and an application in financial econometrics demonstrate competitive performance of the new method an extension to the non parametric factor model is also discussed

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

the approximability of constraint satisfaction problems we study optimization problems that may be expressed as boolean constraint satisfaction problems an instance of a boolean constraint satisfaction problem is given by m constraints applied to n boolean variables different computational problems arise from constraint satisfaction problems depending on the nature of the underlying constraints as well as on the goal of the optimization task here we consider four possible goals max csp min csp is the class of problems where the goal is to find an assignment maximizing the number of satisfied constraints minimizing the number of unsatisfied constraints max ones min ones is the class of optimization problems where the goal is to find an assignment satisfying all constraints with maximum minimum number of variables set to 1 each class consists of infinitely many problems and a problem within a class is specified by a finite collection of finite boolean functions that describe the possible constraints that may be used tight bounds on the approximability of every problem in max csp were obtained by creignou j comput system sci 51 1995 pp 511 522 in this work we determine tight bounds on the approximability i e the ratio to within which each problem may be approximated in polynomial time of every problem in max ones min csp and min ones combined with the result of creignou this completely classifies all optimization problems derived from boolean constraint satisfaction our results capture a diverse collection of optimization problems such as max 3 sat max cut max clique min cut nearest codeword etc our results unify recent results on the in approximability of these optimization problems and yield a compact presentation of most known results moreover these results provide a formal basis to many statements on the behavior of natural optimization problems that have so far been observed only empirically

a potential reduction algorithm allowing column generation using the dantzig wolfe decomposition technique a potential reduction algorithm allowing column generation for the linear feasibility lf problem is developed the point of departure is a simple containing polytope and its analytic center in each iteration an inequality violated at the current center is selected used to cut the polytope and then used to find the new center for the shrunken polytope the potential value associated with the containing polytope is reduced by a constant at each step and the algorithm is terminated in a polynomial time that depends only on the number and data length of the inequalities generated in the iterative process

constraint relaxation may be perfect abstract networks of constraints are a simple knowledge representation method useful for describing those problems whose solution is required to satisfy several simultaneous constraints the problem of solving a network of constraints with finite domains is np complete the standard solution technique for such networks of constraints is the backtrack search but many relaxation algorithms to be applied before backtracking have been developed they transform a network in an equivalent but more explicit one the idea is to make the backtrack search have a better average time complexity in fact if the network elaborated by the backtrack algorithm is more explicit the algorithm backtracks less in this paper we describe relaxation algorithms as sequences of applications of relaxation rules moreover we define perfect relaxation algorithms as relaxation algorithms which not only return a more explicit network but also exactly solve the given network of constraints by applying every relaxation rule only once finally we characterize a family of classes of networks on which certain perfect relaxation algorithms are very efficient the exact solution of each network in a class is found in linear time

optimal selection of reduced rank estimators of high dimensional matrices we introduce a new criterion the rank selection criterion rsc for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models the corresponding rsc estimator minimizes the frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model the rank of the rsc estimator provides a consistent estimator of the rank of the coefficient matrix in general the rank of our estimator is a consistent estimate of the effective rank which we define to be the number of singular values of the target matrix that are appropriately large the consistency results are valid not only in the classic asymptotic regime when n the number of responses and p the number of predictors stay bounded and m the number of observations grows but also when either or both n and p grow possibly much faster than m we establish minimax optimal bounds on the mean squared errors of our estimators our finite sample performance bounds for the rsc estimator show that it achieves the optimal balance between the approximation error and the penalty term furthermore our procedure has very low computational complexity linear in the number of candidate models making it particularly appealing for large scale problems we contrast our estimator with the nuclear norm penalized least squares nnp estimator which has an inherently higher computational complexity than rsc for multivariate regression models we show that nnp has estimation properties similar to those of rsc albeit under stronger conditions however it is not as parsimonious as rsc we offer a simple correction of the nnp estimator which leads to consistent rank estimation we verify and illustrate our theoretical findings via an extensive simulation study

constraint tightness and looseness versus local and global consistency constraint networks are a simple representation and reasoning framework with diverse applications in this paper we identify two new complementary properties on the restrictiveness of the constraints in a network constraint tightness and constraint looseness and we show their usefulness for estimating the level of local consistency needed to ensure global consistency and for estimating the level of local consistency present in a network in particular we present a sufficient condition based on constraint tightness and the level of local consistency that guarantees that a solution can be found in a backtrack free manner the condition can be useful in applications where a knowledge base will be queried over and over and the preprocessing costs can be amortized over many queries we also present a sufficient condition for local consistency based on constraint looseness that is straightforward and inexpensive to determine the condition can be used to estimate the level of local consistency of a network this in turn can be used in deciding whether it would be useful to preprocess the network before a backtracking search and in deciding which local consistency conditions if any still need to be enforced if we want to ensure that a solution can be found in a backtrack free manner two definitions of local consistency are employed in characterizing the conditions the traditional variable based notion and a recently introduced definition of local consistency called relational consistency

horn functions and submodular boolean functions abstract after providing a simple characterization of horn functions i e those boolean functions that have a horn dnf we study in detail the special class of submodular functions every prime implicant of such a function involves at most one complemented and at most one uncomplemented variable and based on this we give a one to one correspondence between submodular functions and partial preorders reflexive and transitive binary relations and in particular between the nondegenerate acyclic submodular functions and the partially ordered sets there is a one to one correspondence between the roots of a submodular function and the ideals of the associated partial preorder there is also a one to one correspondence between the prime implicants of the dual of the submodular function and the maximal antichains of the associated partial preorder based on these results we give graph theoretic characterizations for all minimum prime dnf representations of a submodular function the problem of recognizing submodular functions in dnf representation is conp complete

on the uniqueness of s functionals and m functionals under nonelliptical distributions the s functionals of multivariate location and scatter including the mve functionals are known to be uniquely defined only at unimodal elliptically symmetric distributions the goal of this paper is to establish the uniqueness of these functionals under broader classes of symmetric distributions we also discuss some implications of the uniqueness of the functionals and give examples of striclty unimodal and symmetric distributions for which the mve functional is not uniquely defined the uniqueness results for the s functionals are obtained by embedding them within a more general class of functionals which we call the m functionals with auxiliary scale the uniqueness results of this paper are then obtained for this class of multivariate functionals besides the s functionals the class of multivariate m functionals with auxiliary scale include the constrained m functionals recently introduced by kent and tyler as well as a new multivariate generalization of yohai s mm functionals

on the minimality and global consistency of row convex constraint networks constraint networks have been shown to be useful in formulating such diverse problems as scene labeling natural language parsing and temporal reasoning given a constraint network we often wish to i find a solution that satisfies the constraints and ii find the corresponding minimal network where the constraints are as explicit as possible both tasks are known to be np complete in the general case task 1 is usually solved using a backtracking algorithm and task ii is often solved only approximately by enforcing various levels of local consistency in this paper we identify a property of binary constraint called row convexity and show its usefulness in deciding when a form of local consistency called path consistency is sufficient to guarantee that a network is both minimal and globally consistent globally consistent networks have the property that a solution can be found without backtracking we show that one can test for the row convexity property efficiently and we show by examining applications of constraint networks discussed in the literature that our results are useful in practice thus we identify a class of binary constraint networks for which we can solve both tasks i and ii efficiently finally we generalize the results for binary constraint networks to networks with nonbinary constraints

combinatorial problems raised from 2 semilattices abstract the constraint satisfaction problem csp provides a general framework for many combinatorial problems in a a bulatov a a krokhin p g jeavons classifying the complexity of constraints using finite algebras siam j comput 34 3 2005 720 742 p g jeavons on the algebraic structure of combinatorial problems theoret comput sci 200 1998 185 204 and then in a a bulatov p g jeavons algebraic structures in combinatorial problems technical report math al 4 2001 technische universitat dresden dresden germany 2001 a new approach to the study of the csp has been developed which uses properties of universal algebras assigned to certain subclasses of the csp such that the time complexity and other properties of subclasses can be derived from the properties of the assigned algebras in this paper we briefly survey this approach and then prove that problem classes corresponding to finite 2 semilattices that is groupoids satisfying the identities x x x x y y x x x y x x y can be solved in polynomial time making use of this result we classify finite conservative groupoids and 4 element algebras with minimal clone of term operations with respect to the complexity of the corresponding csp class

generalising submodularity and horn clauses tractable optimization problems defined by tournament pair multimorphisms the submodular function minimization problem sfm is a fundamental problem in combinatorial optimization and several fully combinatorial polynomial time algorithms have recently been discovered to solve this problem the most general versions of these algorithms are able to minimize any submodular function whose domain is a set of tuples over any totally ordered finite set and whose range includes both finite and infinite values in this paper we demonstrate that this general form of sfm is just one example of a much larger class of tractable discrete optimization problems defined by valued constraints these tractable problems are characterized by the fact that their valued constraints have an algebraic property which we call a tournament pair multimorphism this larger tractable class also includes the problem of satisfying a set of horn clauses horn sat as well as various extensions of this problem to larger finite domains

characterising tractable constraints abstract finding solutions to a binary constraint satisfaction problem is known to be an np complete problem in general but may be tractable in cases where either the set of allowed constraints or the graph structure is restricted this paper considers restricted sets of constraints which are closed under permutation of the labels we identify a set of constraints which gives rise to a class of tractable problems and give polynomial time algorithms for solving such problems and for finding the equivalent minimal network we also prove that the class of problems generated by any set of constraints not contained in this restricted set is np complete

classifying the complexity of constraints using finite algebras many natural combinatorial problems can be expressed as constraint satisfaction problems this class of problems is known to be np complete in general but certain restrictions on the form of the constraints can ensure tractability here we show that any set of relations used to specify the allowed forms of constraints can be associated with a finite universal algebra and we explore how the computational complexity of the corresponding constraint satisfaction problem is connected to the properties of this algebra hence we completely translate the problem of classifying the complexity of restricted constraint satisfaction problems into the language of universal algebra we introduce a notion of tractable algebra and investigate how the tractability of an algebra relates to the tractability of the smaller algebras which may be derived from it including its subalgebras and homomorphic images this allows us to reduce significantly the types of algebras which need to be classified using our results we also show that if the decision problem associated with a given collection of constraint types can be solved efficiently then so can the corresponding search problem we then classify all finite strictly simple surjective algebras with respect to tractability obtaining a dichotomy theorem which generalizes schaefer s dichotomy for the generalized satisfiability problem finally we suggest a possible general algebraic criterion for distinguishing the tractable and intractable cases of the constraint satisfaction problem

hybrid tractability of valued constraint problems the constraint satisfaction problem csp is a central generic problem in computer science and artificial intelligence it provides a common framework for many theoretical problems as well as for many real life applications valued constraint problems are a generalisation of the csp which allow the user to model optimisation problems considerable effort has been made in identifying properties which ensure tractability in such problems in this work we initiate the study of hybrid tractability of valued constraint problems that is properties which guarantee tractability of the given valued constraint problem but which do not depend only on the underlying structure of the instance such as being tree structured or only on the types of valued constraints in the instance such as submodularity we present several novel hybrid classes of valued constraint problems in which all unary constraints are allowed which include a machine scheduling problem constraint problems of arbitrary arities with no overlapping nogoods and the softalldiff constraint with arbitrary unary valued constraints an important tool in our investigation will be the notion of forbidden substructures

optimal cooperation and submodularity for computing potts partition functions with a large number of states the partition function of the q state potts model with random ferromagnetic couplings in the large q limit is generally dominated by the contribution of a single diagram of the high temperature expansion computing this dominant diagram amounts to minimizing a particular submodular function we provide a combinatorial optimization algorithm the optimal cooperation algorithm which works in polynomial time for any lattice the implementation of the method and its running time are also discussed

outlier detection using nonconvex penalized regression this article studies the outlier detection problem from the standpoint of penalized regression in the regression model we add one mean shift parameter for each of the n data points we then apply a regularization favoring a sparse vector of mean shift parameters the usual l1 penalty yields a convex criterion but fails to deliver a robust estimator the l1 penalty corresponds to soft thresholding we introduce a thresholding denoted by based iterative procedure for outlier detection ipod a version based on hard thresholding correctly identifies outliers on some hard test problems we describe the connection between ipod and m estimators our proposed method has one tuning parameter with which to both identify outliers and estimate regression coefficients a data dependent choice can be made based on the bayes information criterion the tuned ipod shows outstanding performance in identifying outliers in various situations compared with other existing approaches in addition ipod is much

a dichotomy theorem for constraint satisfaction problems on a 3 element set the constraint satisfaction problem csp provides a common framework for many combinatorial problems the general csp is known to be np complete however certain restrictions on a possible form of constraints may affect the complexity and lead to tractable problem classes there is therefore a fundamental research direction aiming to separate those subclasses of the csp that are tractable and those which remain np complete schaefer gave an exhaustive solution of this problem for the csp on a 2 element domain in this article we generalise this result to a classification of the complexity of the csp on a 3 element domain the main result states that every subproblem of the csp is either tractable or np complete and the criterion separating them is that conjectured in bulatov et al 2005 and bulatov and jeavons 2001b we also characterize those subproblems for which standard constraint propagation techniques provide a decision procedure finally we exhibit a polynomial time algorithm which for a given set of allowed constraints outputs if this set gives rise to a tractable problem class to obtain the main result and the algorithm we extensively use the algebraic technique for the csp developed in jeavons 1998b bulatov et al 2005 and bulatov and jeavons 2001b

tractable constraints on ordered domains abstract finding solutions to a constraint satisfaction problem is known to be an np complete problem in general but may be tractable in cases where either the set of allowed constraints or the graph structure is restricted in this paper we identify a restricted set of contraints which gives rise to a class of tractable problems this class generalizes the notion of a horn formula in propositional logic to larger domain sizes we give a polynomial time algorithm for solving such problems and prove that the class of problems generated by any larger set of constraints is np complete

a generic arc consistency algorithm and its specializations consistency techniques have been studied extensively in the past as a way of tackling constraint satisfaction problems csp in particular various arc consistency algorithms have been proposed originating from waltz s filtering algorithm and culminating in the optimal algorithm ac 4 of mohr and henderson ac 4 runs in o ed 2 in the worst case where e is the number of arcs or constraints and d is the size of the largest domain being applicable to the whole class of binary csp these algorithms do not take into account the semantics of constraints in this paper we present a new generic arc consistency algorithm ac 5 the algorithm is parameterized on two specified procedures and can be instantiated to reduce to ac 3 and ac 4 more important ac 5 can be instantiated to produce an o ed 2 algorithm for a number of important classes of constraints functional anti functional monotonic and their generalization to functional anti functional and monotonic piecewise constraints we also show that ac 5 has an important application in constraint logic programming over finite domains the kernel of the constraint solver for such a programming language is an arc consistency algorithm for a set of basic constraints we prove that ac 5 in conjuction with node consistency provides a decision procedure for these constraints running in time o ed

reduced rank regression via adaptive nuclear norm penalization we propose an adaptive nuclear norm penalization approach for low rank matrix approximation and use it to develop a new reduced rank estimation method for high dimensional multivariate regression the adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix and it is generally non convex under the natural restriction that the weight decreases with the singular value however we show that the proposed non convex penalized regression method has a global optimal solution obtained from an adaptively soft thresholded singular value decomposition the method is computationally efficient and the resulting solution path is continuous the rank consistency of and prediction estimation performance bounds for the estimator are established for a high dimensional asymptotic regime simulation studies and an application in genetics demonstrate its efficacy

a fast algorithm for the minimum covariance determinant estimator the minimum covariance determinant mcd method of rousseeuw is a highly robust estimator of multivariate location and scatter its objective is to find h observations out of n whose covariance matrix has the lowest determinant until now applications of the mcd were hampered by the computation time of existing algorithms which were limited to a few hundred objects in a few dimensions we discuss two important applications of larger size one about a production process at philips with n 677 objects and p 9 variables and a dataset from astronomy with n 137 256 objects and p 27 variables to deal with such problems we have developed a new algorithm for the mcd called fast mcd the basic ideas are an inequality involving order statistics and determinants and techniques which we call selective iteration and nested extensions for small datasets fast mcd typically finds the exact mcd whereas for larger datasets it gives more accurate results than existing algorithms and is faster by orders

the power of linear programming for general valued csps let d called the domain be a fixed finite set and let gamma called the valued constraint language be a fixed set of functions of the form f d m to mathbb q cup infty where different functions might have different arity m we study the valued constraint satisfaction problem parametrized by gamma denoted by vcsp gamma these are minimization problems given by n variables and the objective function given by a sum of functions from gamma each depending on a subset of the n variables for example if d 0 1 and gamma contains all ternary 0 infty valued functions vcsp gamma corresponds to 3 sat more generally if gamma contains only 0 infty valued functions vcsp gamma corresponds to csp gamma if d 0 1 and gamma contains all ternary 0 1 valued functions vcsp gamma corresponds to min 3 sat in which the goal is to minimize the number of unsatisfied clauses in a 3 cnf instance finite valued constraint languages contain functions that take on only rational values and not infinite values our main result is a precise algebraic characterization of valued constraint languages whose instances can be solved exactly by the basic linear programming relaxation blp for a valued constraint language gamma blp is a decision procedure for gamma if and only if gamma admits a symmetric fractional polymorphism of every arity for a finite valued constraint language gamma blp is a decision procedure if and only if gamma admits a symmetric fractional polymorphism of some arity or equivalently if gamma admits a symmetric fractional polymorphism of arity 2 using these results we obtain tractability of several novel classes of problems including problems over valued constraint languages that are 1 submodular on arbitrary lattices 2 k submodular on arbitrary finite domains 3 weakly and hence strongly tree submodular on arbitrary trees

a sufficient condition for backtrack bounded search backtrack search is often used to solve constraint satisfaction problems a relationship involving the structure of the constraints is described that provides a bound on the backtracking required to advance deeper into the backtrack tree this analysis leads to upper bounds on the effort required for solution of a class of constraint satisfaction problems the solutions involve a combination of relaxation preprocessing and backtrack search the bounds are expressed in terms of the structure of the constraint connections specifically the effort is shown to have a bound exponential in the size of the largest biconnected component of the constraint graph as opposed to the size of the graph as a whole

solving weighted csp by maintaining arc consistency recently a general definition of arc consistency ac for soft constraint frameworks has been proposed by t schiex proc cp 2000 singapore 2000 pp 411 424 in this paper we specialize this definition to weighted csp and introduce two o ed3 enforcing algorithms then we refine the definition and introduce a stronger form of arc consistency ac along with two o n2d2 ed3 algorithms as in the csp case an important application of ac is to combine it with search we empirically demonstrate that a branch and bound algorithm that maintains either ac or ac is a state of the art general solver for weighted csp our experiments cover binary max csp and max sat problems

a faster scaling algorithm for minimizing submodular functions combinatorial strongly polynomial algorithms for minimizing submodular functions have been developed by iwata fleischer and fujishige iff and by schrijver the iff algorithm employs a scaling scheme for submodular functions whereas schrijver s algorithm achieves strongly polynomial bound with the aid of distance labeling subsequently fleischer and iwata have described a push relabel version of schrijver s algorithm to improve its time complexity this paper combines the scaling scheme with the push relabel framework to yield a faster combinatorial algorithm for submodular function minimization the resulting algorithm improves over the previously best known bound by essentially a linear factor in the size of the underlying ground set

complexity analysis of the analytic center cutting plane method that uses multiple cuts we analyze the complexity of the analytic center cutting plane or column generation algorithm for solving general convex problems defined by a separation oracle the oracle is called at the analytic center of a polytope which contains a solution set and is given by the intersection of the linear inequalities previously generated from the oracle if the center is not in the solution set separating hyperplanes will be placed through the center to shrink the containing polytope while the complexity result has been recently established for the algorithm when one cutting plane is placed in each iteration the result remains open when multiple cuts are added moreover adding multiple cuts actually is a key to practical effectiveness in solving many problems and it presents theoretical difficulties in analyzing cutting plane methods in this paper we show that the analytic center cutting plane algorithm with multiple cuts added in each iteration still is a fully polynomial approximation algorithm

decomposing constraint satisfaction problems using database techniques abstract there is a very close relationship between constraint satisfaction problems and the satisfaction of join dependencies in a relational database which is due to a common underlying structure namely a hypergraph by making that relationship explicit we are able to adapt techniques previously developed for the study of relational databases to obtain new results for constraint satisfaction problems in particular we prove that a constraint satisfaction problem may be decomposed into a number of subproblems precisely when the corresponding hypergraph satisfies a simple condition we show that combining this decomposition approach with existing algorithms can lead to a significant improvement in efficiency

reduction operations in fuzzy or valued constraint satisfaction in the constraint satisfaction problem csp knowledge about a relation on n variables is given in the form of constraints on subsets of the variables a solution to the csp is simply an instantiation of the n variables which satisfies all the constraints the fuzzy constraint satisfaction problem fcsp is a generalisation of the csp to the case in which the constraints are not categorical but represent preferences in the form of fuzzy sets the fcsp is then an optimisation problem in the space of all possible instantiations of the n variables it has potentially many more applications in real world problems than the csp when the aim is to maximise the value of the most violated constraint then the fcsp can be solved by solving a logarithmic number of csps fuzzy k consistency can even be established by an algorithm whose worst case complexity is identical to that of an optimal k consistency algorithm for csps when the operator used to aggregate constraint values is strictly monotonic as is the case in max csp for example the classic operation of arc consistency has to be completely redefined we show that an ordered version of arc consistency exists for all fcsps with a strictly monotonic aggregation operator and that in the case of max csp it can be established by an algorithm whose worst case complexity is identical to that of the optimal arc consistency algorithm for csps neighbourhood substitution in csps can be generalised to fuzzy neighbourhood substitution in fcsps whether the aggregation operator is strictly monotonic or idempotent we give an algorithm for applying fuzzy neighbourhood substitutions until convergence based on the corresponding algorithm for csps

an optimal k consistency algorithm abstract this paper generalizes the arc consistency algorithm of mohr and henderson 4 and the path consistency algorithm of han and lee 2 to a k consistency algorithm arc consistency and path consistency being 2 consistency and 3 consistency respectively the algorithm is a development of freuder s synthesis algorithm 1 it simultaneously establishes i consistency for each 1 i k it has worst case time and space complexity which is optimal when k is a constant and almost optimal for all other values of k in the case that all order i constraints exist for all 1 i n this algorithm is a solution to the consistent labeling problem with almost optimal worst case time and space complexity

minimizing symmetric submodular functions we describe a purely combinatorial algorithm which given a submodular set functionf on a finite setv finds a nontrivial subseta ofv minimizingf a f v a this algorithm an extension of the nagamochi ibaraki minimum cut algorithm as simplified by stoer and wagner m stoer f wagner a simple min cut algorithm proceedings of the european symposium on algorithms esa 94 lncs 855 springer berlin 1994 pp 141 147 and by frank a frank on the edge connectivity algorithm of nagamochi and ibaraki laboratoire artemis imag universite j fourier grenbole 1994 minimizes any symmetric submodular function using o v 3 calls to a function value oracle 1998 the mathematical programming society inc published by elsevier science b v

the complexity of soft constraint satisfaction over the past few years there has been considerable progress in methods to systematically analyse the complexity of constraint satisfaction problems with specified constraint types one very powerful theoretical development in this area links the complexity of a set of constraints to a corresponding set of algebraic operations known as polymorphisms in this paper we extend the analysis of complexity to the more general framework of combinatorial optimisation problems expressed using various forms of soft constraints we launch a systematic investigation of the complexity of these problems by extending the notion of a polymorphism to a more general algebraic operation which we call a multimorphism we show that many tractable sets of soft constraints both established and novel can be characterised by the presence of particular multimorphisms we also show that a simple set of np hard constraints has very restricted multimorphisms finally we use the notion of multimorphism to give a complete classification of complexity for the boolean case which extends several earlier classification results for particular special cases

the complexity of conservative valued csps we study the complexity of valued constraint satisfaction problems vcsps parametrized by a constraint language a fixed set of cost functions over a finite domain an instance of the problem is specified by a sum of cost functions from the language and the goal is to minimize the sum under the unique games conjecture the approximability of finite valued vcsps is well understood see raghavendra 2008 however there is no characterization of finite valued vcsps let alone general valued vcsps that can be solved exactly in polynomial time thus giving insights from a combinatorial optimization perspective we consider the case of languages containing all possible unary cost functions in the case of languages consisting of only 0 valued cost functions i e relations such languages have been called conservative and studied by bulatov 2003 2011 and recently by barto 2011 since we study valued languages we call a language conservative if it contains all finite valued unary cost functions the computational complexity of conservative valued languages has been studied by cohen et al 2006 for languages over boolean domains by deineko et al 2008 for 0 1 valued languages a k a max csp and by takhanov 2010a for 0 valued languages containing all finite valued unary cost functions a k a min cost hom we prove a schaefer like dichotomy theorem for conservative valued languages if all cost functions in the language satisfy a certain condition specified by a complementary combination of stp and mjn multimorphisms then any instance can be solved in polynomial time via a new algorithm developed in this article otherwise the language is np hard this is the first complete complexity classification of general valued constraint languages over non boolean domains it is a common phenomenon that complexity classifications of problems over non boolean domains are significantly harder than the boolean cases the polynomial time algorithm we present for the tractable cases is a generalization of the submodular minimization problem and a result of cohen et al 2008 our results generalize previous results by takhanov 2010a and a subset of results by cohen et al 2006 and deineko et al 2008 moreover our results do not rely on any computer assisted search as in deineko et al 2008 and provide a powerful tool for proving hardness of finite valued and general valued languages

a build down scheme for linear programming we propose a build down scheme for karmarkar s algorithm and the simplex method for linear programming the scheme starts with an optimal basis candidate set including all columns of the constraint matrix then constructs a dual ellipsoid containing all optimal dual solutions a pricing rule is developed for checking whether or not a dual hyperplane corresponding to a column intersects the containing ellipsoid if the dual hyperplane has no intersection with the ellipsoid its corresponding column will not appear in any of the optimal bases and can be eliminated from as these methods iterate is eventually built down to a set that contains only the optimal basic columns

the approximability of three valued max csp in the maximum constraint satisfaction problem max csp one is given a finite collection of possibly weighted constraints on overlapping sets of variables and the goal is to assign values from a given domain to the variables so as to maximize the number or the total weight for the weighted case of satisfied constraints this problem is np hard in general and therefore it is natural to study how restricting the allowed types of constraints affects the approximability of the problem it is known that every boolean that is two valued max csp with a finite set of allowed constraint types is either solvable exactly in polynomial time or else apx complete and hence can have no polynomial time approximation scheme unless p np it has been an open problem for several years whether this result can be extended to non boolean max csp which is much more difficult to analyze than the boolean case in this paper we make the first step in this direction by establishing this result for max csp over a three element domain moreover we present a simple description of all polynomial time solvable cases of our problem this description uses the well known algebraic combinatorial property of supermodularity we also show that every hard three valued max csp contains in a certain specified sense one of the two basic hard max csps which are the maximum k colorable subgraph problems for k 2 3

closure properties of constraints many combinatorial search problems can be expressed as constraint satisfaction problems and this class of problems is known to be np complete in general in this paper we investigate the subclasses that arise from restricting the possible constraint types we first show that any set of constraints that does not give rise to an np complete class of problems must satisfy a certain type of algebraic closure condition we then investigate all the different possible forms of this algebraic closure property and establish which of these are sufficient to ensure tractability as examples we show that all known classes of tractable constraints over finite domains can be characterized by such an algebraic closure property finally we describe a simple computational procedure that can be used to determine the closure properties of a given set of constraints this procedure involves solving a particular constraint satisfaction problem which we call an indicator problem

robust reduced rank regression in high dimensional multivariate regression problems enforcing low rank in the coefficient matrix offers effective dimension reduction which greatly facilitates parameter estimation and model interpretation however commonly used reduced rank methods are sensitive to data corruption as the low rank dependence structure between response variables and predictors is easily distorted by outliers we propose a robust reduced rank regression approach for joint modelling and outlier detection the problem is formulated as a regularized multivariate regression with a sparse mean shift parameterization which generalizes and unifies some popular robust multivariate methods an efficient thresholding based iterative procedure is developed for optimization we show that the algorithm is guaranteed to converge and that the coordinatewise minimum point produced is statistically accurate under regularity conditions our theoretical investigations focus on non asymptotic robust analysis demonstrating that joint rank reduction and outlier detection leads to improved prediction accuracy in particular we show that redescending formula see text functions can essentially attain the minimax optimal error rate and in some less challenging problems convex regularization guarantees the same low error rate the performance of the proposed method is examined through simulation studies and real data examples

a polynomial time primal dual affine scaling algorithm for linear and convex quadratic programming and its power series extension we describe an algorithm for linear and convex quadratic programming problems that uses power series approximation of the weighted barrier path that passes through the current iterate in order to find the next iterate if r 1 is the order of approximation used we show that our algorithm has time complexity o n 1 1 r l 1 1 r iterations and o n3 n2r arithmetic operations per iteration where n is the dimension of the problem and l is the size of the input data when r 1 we show that the algorithm can be interpreted as an affine scaling algorithm in the primal dual setup

comments on mohr and henderson s path consistency algorithm abstract mohr and henderson have presented new algorithms for arc and path consistency in 1 though the underlying ideas of their algorithms are correct the path consistency algorithm pc 3 is in error in this paper we point out the errors in this algorithm and give a correct one the time complexity and space complexity of the revised algorithm are also analyzed

on binary constraint problems the concepts of binary constraint satisfaction problems can be naturally generalized to the relation algebras of tarski the concept of path consistency plays a central role algorithms for path consistency can be implemented on matrices of relations and on matrices of elements from a relation algebra we give an example of a 4 by 4 matrix of infinite relations on which on iterative local path consistency algorithm terminates we give a class of examples over a fixed finite algebra on which all iterative local algorithms whether parallel or sequential must take quadratic time specific relation algebras arising from interval constraint problems are also studied the interval algebra the point algebra and the containment algebra

fast parallel constraint satisfaction a constraint satisfaction problem csp involves searching for an assignment of values to a given set of variables so that the values assigned satisfy a given set of constraints the general csp is np complete to confront the intractability of the general csp relaxation procedures have been devised instead of searching for a globally consistent assignment of values to the variables try to restrict the domain of values of each variable in a way that ensures local consistency only the relaxation procedures are efficient but have been proved to be inherently sequential in this paper we define a class of csps for which a global solution can be found by a fast parallel algorithm no relaxation preprocessing is needed for the parallel algorithm to work the result is motivated from the problem of labelling a 2 d line drawing of a 3 d object by the clowes huffman malik labelling scheme an important application of csp in computer vision for such a labelling csp the constraint graph can be general but the constraint relations are usually of the type we call implicational it is shown here that a csp with this type of constraint relations and no restrictions on its graph can be solved by an efficient i e with polynomial time complexity sequential algorithm also it is shown that it can be solved by a fast parallel algorithm that executes in time o log3n with o m n3 log n processors on a exlusive read exclusive write parallel random access machine n is the number of variables and m is the number of constrain relations the constraint relations may have arity more than two

on the algebraic structure of combinatorial problems abstract we describe a general algebraic formulation for a wide range of combinatorial problems including satisfiability graph colorability and graph isomorphism in this formulation each problem instance is represented by a pair of relational structures and the solutions to a given instance are homomorphisms between these relational structures the corresponding decision problem consists of deciding whether or not any such homomorphisms exist we then demonstrate that the complexity of solving this decision problem is determined in many cases by simple algebraic properties of the relational structures involved this result is used to identify tractable subproblems of satisfiability and to provide a simple test to establish whether a given set of boolean relations gives rise to one of these tractable subproblems

constraints consistency and closure although the constraint satisfaction problem is np complete in general a number of constraint classes have been identified for which some fixed level of local consistency is sufficient to ensure global consistency in this paper we describe a simple algebraic property which characterises all possible constraint types for which strong k consistency is sufficient to ensure global consistency for each k 2 we give a number of examples to illustrate the application of this result

a combinatorial strongly polynomial algorithm for minimizing submodular functions this paper presents a combinatorial polynomial time algorithm for minimizing submodular functions answering an open question posed in 1981 by grotschel lovasz and schrijver the algorithm employs a scaling scheme that uses a flow in the complete directed graph on the underlying set with each arc capacity equal to the scaled parameter the resulting algorithm runs in time bounded by a polynomial in the size of the underlying set and the length of the largest absolute function value the paper also presents a strongly polynomial version in which the number of steps is bounded by a polynomial in the size of the underlying set independent of the function values

the computational structure of monotone monadic snp and constraint satisfaction a study through datalog and group theory this paper starts with the project of finding a large subclass of np which exhibits a dichotomy the approach is to find this subclass via syntactic prescriptions while the paper does not achieve this goal it does isolate a class of problems specified by monotone monadic snp without inequality which may exhibit this dichotomy we justify the placing of all these restrictions by showing essentially using ladner s theorem that classes obtained by using only two of the above three restrictions do not show this dichotomy we then explore the structure of this class we show that all problems in this class reduce to the seemingly simpler class csp we divide csp into subclasses and try to unify the collection of all known polytime algorithms for csp problems and extract properties that make csp problems np hard this is where the second part of the title a study through datalog and group theory comes in we present conjectures about this class which would end in showing the dichotomy

arc consistency for soft constraints the notion of arc consistency plays a central role in constraint satisfaction r dechter constraint processing morgan kaufmann san mateo ca 2003 it is known since the introduction of valued and semi ring constraint networks in 1995 that the notion of local consistency can be extended to constraint optimisation problems defined by soft constraint frameworks based on an idempotent cost combination operator this excludes non idempotent operators such as which define problems which are very important in practical applications such as max csp where the aim is to minimise the number of violated constraints in this paper we show that using a weak additional axiom satisfied by most existing soft constraints proposals it is possible to define a notion of soft arc consistency that extends the classical notion of arc consistency and this even in the case of non idempotent cost combination operators a polynomial time algorithm for enforcing this soft arc consistency exists and its space and time complexities are identical to that of enforcing arc consistency in csps when the cost combination operator is strictly monotonic for example max csp a directional version of arc consistency first introduced by m c cooper reduction operations in fuzzy or valued constraint satisfaction fuzzy sets and systems 134 3 2003 311 342 is potentially even stronger than the non directional version since it allows non local propagation of penalties we demonstrate the utility of directional arc consistency by showing that it not only solves soft constraint problems on trees but that it also implies a form of local optimality which we call arc irreducibility

a faster strongly polynomial time algorithm for submodular function minimization we consider the problem of minimizing a submodular function f defined on a set v with n elements we give a combinatorial algorithm that runs in o n 5eo n 6 time where eo is the time to evaluate f s for some s subseteq v this improves the previous best strongly polynomial running time by more than a factor of n we also extend our result to ring families

from local to global consistency abstract in reasoning tasks involving the maintenance of consistent databases so called qqconstraint networks q q it is customary to enforce local consistency conditions in order to simplify the subsequent construction of a globally coherent model of the data in this paper we present a relationship between the sizes of the variables domains the constraints arity and the level of local consistency sufficient to ensure global consistency based on these parameters a new tractability classification of constraint networks is presented we also show based on this relationship that any relation on bi valued variables which is not representable by a network of binary constraints cannot be represented by networks with any number of hidden variables

a qp free globally convergent locally superlinearly convergent algorithm for inequality constrained optimization an algorithm is proposed for the minimization of a smooth function subject to smooth inequality constraints unlike sequential quadratic programming type methods this algorithm does not involve the solution of quadratic programs but merely that of linear systems of equations locally the iteration can be viewed as a perturbation of a quasi newton iteration on both the primal and dual variables for the solution of the equalities in the kuhn tucker first order conditions of optimality it is observed that provided the current iterate is feasible and the current multiplier estimates are strictly positive the primal component of the quasi newton direction is a direction of descent for the objective function this fact is used to induce global convergence without the need of a surrogate merit function a careful bending of the search direction prevents any maratos like effect and local superlinear convergence is proven while the algorithm requires that an initial feasible point be available the succe

constraint reduction for linear programs with many inequality constraints consider solving a linear program in standard form where the constraint matrix a is m times n with n gg m gg 1 such problems arise for example as the result of finely discretizing a semi infinite program the cost per iteration of typical primal dual interior point methods on such problems is o m 2n we propose to reduce that cost by replacing the normal equation matrix ad 2a t where d is a diagonal matrix with a reduced version of same dimension a qd q 2a q t where q is an index set including the indices of m most nearly active or most violated dual constraints at the current iterate with m geq m a prescribed integer this can result in a speedup of close to n q at each iteration promising numerical results are reported for constraint reduced versions of a dual feasible affine scaling algorithm and of mehrotra s predictor corrector method s mehrotra it siam j optim 2 1992 pp 575 601 in particular while it could be expected that neglecting a large portion of the constraints especially at early iterations may result in a significant deterioration of the search direction it appears that the total number of iterations typically remains essentially constant as the size of the reduced constraint set is decreased down to some threshold in some cases this threshold is a small fraction of the total set in the case of the affine scaling algorithm global convergence and local quadratic convergence are proved