oracles for deterministic versus alternating classes we construct oracles that force all possible relationships between textit np and textit exp k textit dtime 2 o n k we generalize these results to obtain a theorem about oracles that force relationships between deterministic and nondeterministic and alternating classes from which many corollaries follow the corollaries are interesting because we compare a powerful type of machine e g nondeterministic alternating to a less powerful type of machine that can use more time one of our corollaries is that the result textit dtime n log n subseteq sigma 2 text time n does not relativize

an aggregate subgradient method for nonsmooth convex minimization a class of implementable algorithms is described for minimizing any convex not necessarily differentiable functionf of several variables the methods require only the calculation off and one subgradient off at designated points they generalize lemarechal s bundle method more specifically instead of using all previously computed subgradients in search direction finding subproblems that are quadratic programming problems the methods use an aggregate subgradient which is recursively updated as the algorithms proceed each algorithm yields a minimizing sequence of points and iff has any minimizers then this sequence converges to a solution of the problem particular members of this algorithm class terminate whenf is piecewise linear the methods are easy to implement and have flexible storage requirements and computational effort per iteration that can be controlled by a user

a uniform approach to define complexity classes abstract complexity classes are usually defined by referring to computation models and by putting suitable restrictions on them following this approach many proofs of results are tightly bound to the characteristics of the computation model and of its restrictions and therefore they sometimes hide the essential properties which insure the obtained results in order to obtain more general results a uniform family of computation models which encompasses most of the complexity classes of interest is introduced as a first initial set of results derivable from the proposed approach we will give a sufficient and necessary condition for proving separations of relativized complexity classes a characterization of complexity classes with complete languages and a sufficient condition for proving strong separations of relativized complexity classes examples of applications of these results to some specific complexity classes are then given additional results related to separations by sparse oracles can be found in bovet 1991

resource bounded immunity and simplicity revisiting the 30 years old notions of resource bounded immunity and simplicity we investigate the structural characteristics of various immunity notions strong immunity almost immunity and hyperimmunity as well as their corresponding simplicity notions we also study limited immunity and simplicity called k immunity and feasible k immunity and their simplicity notions finally we propose the k immune hypothesis as a working hypothesis that guarantees the existence of simple sets in np

the boolean hierarchy i structural properties in this paper we study the complexity of sets formed by boolean operations union intersection and complement on np sets these are the sets accepted by trees of hardware with np predicates as leaves and together these form the boolean hierarchy we present many results about the structure of the boolean hierarchy separation and immunity results natural complete languages and structural asymmetries between complementary classes we show that in some relativized worlds the boolean hierarchy is infinite and that for every k there is a relativized world in which the boolean hierarchy extends exactly k levels we prove natural languages variations of vertex cover complete for the various levels of the boolean hierarchy we show the following structural asymmetry though no set in the boolean hierarchy is text d text p immune there is a relativized world in which the boolean hierarchy contains text cod text p immune sets thus this paper explores the structural properties of the

limitations on separating nondeterministic complexity classes if the time bounds defining two nondeterministic complexity classes are too close for separation by the two known techniques then they are almost too close for separation by any relativizable technique proof of an analogous result for space would be a major breakthrough implying operatorname nspace log n operatorname dspace log n

sparse sets and collapse of complexity classes in this paper a new upward separation technique is developed it is applied to prove that for a class of functions f a separation dtime f ntime f can be characterized by the existence of not only polynomially sparse sets in certain complexity classes as a consequence a solution of an open question of j hartmanis 1983 inform process lett 16 55 60 is obtained there is an no log n sparse set in np p iff c 1ntime 2c n c 1dtime 2c n further we prove that there is an no log n sparse set in np conp iff c 1ntime 2c n c 1contime 2c n the technique also allows us to characterize the existence of sets of different densities in np p by the existence of slightly denser sets in ntime f dtime f for certain classes of functions f for example there is an no log n sparse set in np p iff there is an no log n 3 sparse set in c 1ntime nclog n c 1dtime nclog n the end of the paper is devoted to limitations of the technique

sparse sets in np p relativizations we construct an oracle relative to which bf p ne bf np and there are no sparse sets in bf np bf p the well known construction of baker gill and solovay siam j comput 4 1975 pp 431 442 gives an oracle relative to which there is a sparse set in bf np bf p together these results show that simple modifications of conventional proof techniques cannot establish whether or not sparse sets exist in bf np bf p even if one assumes bf p ne bf np

bundle methods for regularized risk minimization a wide variety of machine learning problems can be described as minimizing a regularized risk functional with different algorithms using different notions of risk and different regularizers examples include linear support vector machines svms gaussian processes logistic regression conditional random fields crfs and lasso amongst others this paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems it can be parallelized on a cluster of workstations allows for data locality and can deal with regularizers such as l1 and l2 penalties in addition to the unified framework we present tight convergence bounds which show that our algorithm converges in o 1 e steps to e precision for general convex problems and in o log 1 e steps for continuously differentiable problems we demonstrate the performance of our general purpose solver on a variety of publicly available data sets

faster algorithms for max product message passing maximum a posteriori inference in graphical models is often solved via message passing algorithms such as the junction tree algorithm or loopy belief propagation the exact solution to this problem is well known to be exponential in the size of the maximal cliques of the triangulated model while approximate inference is typically exponential in the size of the model s factors in this paper we take advantage of the fact that many models have maximal cliques that are larger than their constituent factors and also of the fact that many factors consist only of latent variables i e they do not depend on an observation this is a common case in a wide variety of applications that deal with grid tree and ring structured models in such cases we are able to decrease the exponent of complexity for message passing by 0 5 for both exact and approximate inference we demonstrate that message passing operations in such models are equivalent to some variant of matrix multiplication in the tropical semiring for which we offer an o n2 5 expected case solution

tally np sets and easy census functions abstract we study the question of whether every p set has an easy i e polynomial time computable census function we characterize this question in terms of unlikely collapses of language and function classes such as p1 fp where p1 is the class of functions that count the witnesses for tally np sets we prove that every p1ph function can be computed in fp p1 p1 consequently every p set has an easy census function if and only if every set in the polynomial hierarchy does we show that the assumption p1 fp implies p bpp and ph modk p for each k 2 we also relate a set s property of having an easy census function to other well studied properties of sets such as rankability and scalability the closure of the rankable sets under p isomorphisms finally we prove that it is no more likely that the census function of any set in p can be approximated more precisely can be n enumerated in time n for fixed and than that it can be precisely computed in polynomial time

polynomial terse sets abstract let a be a set and k n be such that we wish to know the answers to x1 a x2 a xk a for various k tuples x1 x2 xk if this problem requires k queries to a in order to be solved in polynomial time then a is called polynomial terse or pterse we show the existence of both arbitrarily complex pterse and non pterse sets and that p np iff every np complete set is pterse we also show connections with p immunity p selective p generic sets and the boolean hierarchy in our framework unique satisfiability and a variation of it called ksat is in some sense close to satisfiability

on truth table reducibility to sat we show that polynomial time truth table reducibility via boolean circuits to sat is the same as logspace truth table reducibility via boolean formulas to sat and the same as logspace turing reducibility to sat in addition we prove that a constant number of rounds of parallel queries to sat is equivalent to one round of parallel queries we give an oracle relative to which 2p is not equal to the class of predicates polynomial time truth table reducible to sat

erratum the polynomial time hierarchy collapses if the boolean hierarchy collapses it is shown that if the boolean hierarchy bh collapses then there exists a sparse set s such that text co np subseteq text np s and therefore the polynomial time hierarchy ph col

relativizing time space and time space consider the classes of formal languages specified by nondeterministic acceptors that operate simultaneously within time bounds from a set mathcal t and space bounds from a set mathcal s how large must the time bounds be in order to obtain all of the languages specified by nondeterministic acceptors that operate within space bounds from mathcal s how large must the space bounds be in order to obtain all of the languages specified by nondeterministic acceptors that operate with time bounds from mathcal t the first question is shown to be equivalent with appropriate restrictions on mathcal s and mathcal t to the question of whether it matters if the time bounds apply to all of the steps or only to the steps which query the oracle the second question is shown to be equivalent to the question of whether it matters if the space bounds apply to all of the configurations or only to the configurations in which the oracle is queried these results generalize a more specific result 6

lower bounds for the low hierarchy the low hierarchy in np sc 83 and the extended low hierarchy bbs 86 have been useful in characterizing the complexity of certain interesting classes of sets however until now there has been no way of judging whether or not a given lowness result is the best possible

a bundle newton method for nonsmooth unconstrained minimization an algorithm based on a combination of the polyhedral and quadratic approximation is given for finding stationary points for unconstrained minimization problems with locally lips chitz problem functions that are not necessarily convex or differentiable global convergence of the algorithm is established under additional assumptions it is shown that the algorithm generates newton iterations and that the convergence is superlinear some encouraging numerical experience is reported

oracles that compute values this paper focuses on complexity classes of partial functions that are computed in polynomial time with oracles in npmv the class of all multivalued partial functions that are computable nondeterministically in polynomial time concerning deterministic polynomial time reducibilities it is shown that a multivalued partial function is polynomial time computable with k adaptive queries to npmv if and only if it is polynomial time computable via 2k 1 nonadaptive queries to npmv a characteristic function is polynomial time computable with k adaptive queries to npmv if and only if it is polynomial time computable with k adaptive queries to np unless the boolean hierarchy collapses for every k k adaptive nonadaptive queries to npmv are different than k 1 adaptive nonadaptive queries to npmv nondeterministic reducibilities lowness and the difference hierarchy over npmv are also studied the difference hierarchy for partial functions does not collapse unless the boolean hierarchy collapses but surprisingly the levels of the difference and bounded query hierarchies do not interleave as is the case for sets unless the polynomial hierarchy collapses

fine hierarchies and m reducibilities in theoretical computer science this is a survey of results about versions of fine hierarchies and many one reducibilities that appear in different parts of theoretical computer science these notions and related techniques play a crucial role in understanding complexity of finite and infinite computations we try not only to present the corresponding notions and facts from the particular fields but also to identify the unifying notions techniques and ideas

expressiveness of concept expressions in first order description logics we introduce a method for characterizing the expressive power of concept expressions in first order description logics the method is essentially model theoretic in nature in that it gives preservation results uniquely identifying a wide range of description logics as fragments of first order logic the languages studied in the paper all belong to the well known email protected email protected and plane1d 49c hierarchies

a simple label switching algorithm for semisupervised structural svms in structured output learning obtaining labeled data for real world applications is usually costly while unlabeled examples are available in abundance semisupervised structured classification deals with a small number of labeled examples and a large number of unlabeled structured data in this work we consider semisupervised structural support vector machines with domain constraints the optimization problem which in general is not convex contains the loss terms associated with the labeled and unlabeled examples along with the domain constraints we propose a simple optimization approach that alternates between solving a supervised learning problem and a constraint matching problem solving the constraint matching problem is difficult for structured prediction and we propose an efficient and effective label switching method to solve it the alternating optimization is carried out within a deterministic annealing framework which helps in effective constraint matching and avoiding poor local minima which are not very useful the algorithm is simple and easy to implement further it is suitable for any structured output learning problem where exact inference is available experiments on benchmark sequence labeling data sets and a natural language parsing data set show that the proposed approach though simple achieves comparable generalization performance

defying upward and downward separation upward and downward separation results link the collapse of small and large classes and are a standard tool in complexity theory we study the limitations of upward and downward separation pp we show that the exponential time limited nondeterminism hierarchy does not robustly possess downward separation we show that probabilistic classes do not robustly possess hartmanis immerman sewelson upward separation though np is known to robustly possess hartmanis immerman sewelson upward separation we show that np does not robustly possess hartmanis immerman sewelson upward separation with respect to strong immunity separation on the other hand we provide a structural sufficient condition for upward separation

sparse sets in np p exptime versus nexptime this paper investigates the structural properties of sets in np p and shows that the computational difficulty of lower density sets in np depends explicitly on the relations between higher deterministic and nondeterministic time bounded complexity classes the paper exploits the recently discovered upward separation method which shows for example that there exist sparse sets in np p if and only if exptime nexptime in addition the paper uses relativization techniques to determine logical possibilities limitations of these proof techniques and exhibits one of the first natural structural differences between relativized np and conp

a downward collapse within the polynomial hierarchy downward collapse also known as upward separation refers to cases where the equality of two larger classes implies the equality of two smaller classes we provide an unqualified downward collapse result completely within the polynomial hierarchy in particular we prove that for k 2 if rm p sigma p k 1 rm p sigma p k 2 then sigma p k pi p k rm ph we extend this to obtain a more general downward collapse result

structure in approximation classes the study of the approximability properties of np hard optimization problems has recently made great advances mainly due to the results obtained in the field of proof checking the last important breakthrough proves the apx completeness of several important optimization problems and thus reconciles two distinct views of approximation classes syntactic and computational s khanna et al in proc 35th ieee symp on foundations of computer science ieee computer society press los alamitos ca 1994 pp 819 830 in this paper we obtain new results on the structure of several computationally defined approximation classes in particular after defining a new approximation preserving reducibility to be used for as many approximation classes as possible we give the first examples of natural npo complete problems and the first examples of natural apx intermediate problems moreover we state new connections between the approximability properties and the query complexity of npo problems

np trees and carnap s modal logic in this paper we consider problems and complexity classes definable by interdependent queries to an oracle in np how the queries depend on each other is specified by a directed graph g we first study the class of problems where g is a general dag and show that this class coincides with d p 2 we then consider the class where g is a tree our main result states that this class is identical to p np o log n the class of problems solvable in polynomial time with a logarithmic number of queries to an oracle in np this result has interesting applications in the fields of modal logic and artificial intelligence in particular we show that the following problems are all p np o log n complete validity checking of formulas in carnap s modal logic checking whether a formula is almost surely valid over finite structures in modal logics k t and s4 a problem recently considered by halpern and kapron 1992 and checking whether a formula belongs to the stable set of beliefs generated by a propositional theory we generalize the case of dags to the case where g is a general possibly cyclic directed graph of np oracle queries and show that this class corresponds to p p 2 we show that such graphs are easily expressible in autoepistemic logic finally we generalize our complexity results to higher classes of the polynomial time hierarchy

incorporating the loss function into discriminative clustering of structured outputs clustering using the hilbert schmidt independence criterion cluhsic is a recent clustering algorithm that maximizes the dependence between cluster labels and data observations according to the hilbert schmidt independence criterion hsic it is unique in that structure information on the cluster outputs can be easily utilized in the clustering process however while the choice of the loss function is known to be very important in supervised learning with structured outputs we will show in this paper that cluhsic is implicitly using the often inappropriate zero one loss we propose an extension called cluhsical which stands for clustering using hsic and loss which explicitly considers both the output dependency and loss function its optimization problem has the same form as cluhsic except that its partition matrix is constructed in a different manner experimental results on a number of datasets with structured outputs show that cluhsical often outperforms cluhsic in terms of both structured loss and clustering accuracy

propositional circumscription and extended closed world reasoning are p p 2 complete abstract circumscription and the closed world assumption with its variants are well known nonmonotonic techniques for reasoning with incomplete knowledge their complexity in the propositional case has been studied in detail for fragments of propositional logic one open problem is whether the deduction problem for arbitrary propositional theories under the extended closed world assumption or under circumscription is p 2 complete i e complete for a class of the second level of the polynomial hierarchy we answer this question by proving these problems p 2 complete and we show how this result applies to other variants of closed world reasoning

on sets truth table reducible to sparse sets we study sets that are truth table reducible to sparse sets in polynomial time the principal results are as follows 1 for every integer k 0 there is a set l and a sparse set s such that l

efficient algorithms for exact inference in sequence labeling svms the task of structured output prediction deals with learning general functional dependencies between arbitrary input and output spaces in this context two loss sensitive formulations for maximum margin training have been proposed in the literature which are referred to as margin and slack rescaling respectively the latter is believed to be more accurate and easier to handle nevertheless it is not popular due to the lack of known efficient inference algorithms therefore margin rescaling which requires a similar type of inference as normal structured prediction is the most often used approach focusing on the task of label sequence learning we here define a general framework that can handle a large class of inference problems based on hamming like loss functions and the concept of decomposability for the underlying joint feature map in particular we present an efficient generic algorithm that can handle both rescaling approaches and is guaranteed to find an optimal solution in polynomial time

relativized polynomial time hierarchies having exactly k levels it is proved that for every integer k geqq 0 there is an oracle a k relative to which the polynomial time hierarchy collapses so that it has exactly k levels furthermore sets b k and c k may be constructed so that relative to b k the polynomial time hierarchy has exactly k levels and the class pspace coincides with the polynomial time hierarchy and relative to c k the polynomial time hierarchy has exactly k levels and the class pspace is different from the polynomial time hierarchy

downward translations of equality in this paper we construct oracles relative to which dtime t n equals ntime t n and dtime t n does not equal ntime t n for t n sufficiently smaller than t n a stronger result than this is also obtained though for fewer t n expressedin two parts for t n 2n there is an oracle relative to which dtime t n equals ntime t n and ntime 2n contains a set not in dtime t n for any t n growing more slowly than t n for t n 22n 0 1 there is an oracle relative to which dtime t n equals ntime t n and ntime log t n contains a set not in dtime t n for t n growing more slowly than t n these results expand on those obtained by dekhtyar 1976 book et al 1982 and allender 1989

unambiguous computation boolean hierarchies and sparse turing complete sets this paper studies for up two topics that have been intensely studied for np boolean hierarchies and the consequences of the existence of sparse turing complete sets unfortunately as is often the case the results for np draw on special properties of np that do not seem to carry over straightforwardly to up for example it is known for np and more generally for any class containing sigma and emptyset and closed under union and intersection that the symmetric difference hierarchy the boolean hierarchy and the boolean closure all are equal we prove that closure under union is not needed for this claim for any class cal k that contains sigma and emptyset and is closed under intersection e g up us and fewp the symmetric difference hierarchy over cal k the boolean hierarchy over cal k and the boolean closure of cal k all are equal on the other hand we show that two hierarchies the hausdorff hierarchy and the nested difference hierarchy which in the np case are equal to the boolean closure fail to be equal for the up case in some relativized worlds regarding sparse turing complete sets for up we prove that if up has sparse turing complete sets then the levels of the unambiguous polynomial hierarchy are simpler than one would otherwise expect they collapse one level in terms of their location in the promise unambiguous polynomial hierarchy we obtain related results under the weaker assumption that up has sparse turing hard sets

estimating the support of a high dimensional distribution suppose you are given some data set drawn from an underlying probability distribution p and you want to estimate a simple subset s of input space such that the probability that a test point drawn from p lies outside of s equals some a priori specified value between 0 and 1 we propose a method to approach this problem by trying to estimate a function f that is positive on s and negative on the complement the functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data it is regularized by controlling the length of the weight vector in an associated feature space the expansion coefficients are found by solving a quadratic programming problem which we do by carrying out sequential optimization over pairs of input patterns we also provide a theoretical analysis of the statistical performance of our algorithm the algorithm is a natural extension of the support vector algorithm to the case of unlabeled data

bounded queries to sat and the boolean hierarchy we study the complexity of decision problems that can be solved by a polynomial time turing machine that makes a bounded number of queries to an np oracle depending on whether we allow some queries to depend on the results of other queries we obtain two probably different hierarchies we present several results relating the bounded np query hierarchies to each other and to the boolean hierarchy we also consider the similarly defined hierarchies of functions that can be computed by a polynomial time turing machine that makes a bounded number of queries to an np oracle we present relations among these two hierarchies and the boolean hierarchy in particular we show for all k that there are functions computable with 2k parallel queries to an np set that are not computable in polynomial time with k serial queries to any oracle unless p np as a corollary k 1 parallel queries to an np set allow us to compute more functions than are computable with only k parallel queries to an np set unless p np the same is true of serial queries similar results hold for all tt self reducible sets using a mind change technique we show that 2k 1 parallel queries to an np set allow us to accept in polynomial time exactly the same sets as can be accepted in polynomial time with k serial queries to an np set in fact the same is true for any class in place of np that is closed under polynomial time positive bounded truth table reductions this contrasts with the expected result for function computations with an np oracle beigel 1988 in addition we show that the boolean hierarchy and the bounded query hierarchies of languages either stand or collapse together finally we show that if the boolean hierarchy collapses to any level but the zeroth deterministic polynomial time then for all k there are functions computable in polynomial time with k parallel queries to an np set that are not computable in polynomial time with k 1 serial queries to any set np complete sets are p superterse

restricted information from nonadaptive queries to np we investigate classes of sets that can be decided by bounded truth table reductions to an np set in which evaluators donothave full access to the answers to the queries but get only restricted information such as the number of queries that are in the oracle set or even just this number modulom for somem 2 we also investigate the case in which evaluators are nondeterministic we show that when we vary the information that the evaluators get this can change the resulting power of the evaluators we locate all these classes within levels of the boolean hierarchy which allows us to compare the complexity of such classes

extending downward collapse from 1 versus 2 queries to m versus m 1 queries the top part of figure 1 1 shows some classes from the truth table bounded query and boolean hierarchies it is well known that if either of these hierarchies collapses at a given level then all higher levels of that hierarchy collapse to that same level this is a standard upward translation of equality that has been known for over a decade the issue of whether these hierarchies can translate equality downwards has proven vastly more challenging in particular with regard to figure 1 1 consider the following claim psigkmtt psigkmponett implies diffmsigk codiffmsigk bh sigmak this claim if true says that equality translates downwards between levels of the bounded query hierarchy and the boolean hierarchy levels that before the fact are immediately below them until recently it was not known whether ever held except for the degenerate cases m 0 and k 0 then hemaspaandra hemaspaandra and hempel siam j comput 28 1999 pp 383 393 proved that holds for all m for k 2 buhrman and fortnow j comput system sci 59 1999 pp 182 199 then showed that when k 2 holds for the case m 1 in this paper we prove that for the case k 2 holds for all values of m since there is an oracle relative to which for k 1 holds for all m fails see buhrman and fortnow our achievement of the k 2 case cannot be strengthened to k 1 by any relativizable proof technique the new downward translation we obtain also tightens the collapse in the polynomial hierarchy implied by a collapse in the bounded query hierarchy of the second level of the polynomial hierarchy

complexity classes without machines on complete languages for up this paper develops techniques for studying complexity classes that are not covered by known recursive enumerations of machines often counting classes probabilistic classes and intersection classes lack such enumerations concentrating on the counting class up we show that there are relativizations for which upa has no complete languages and other relativizations for which pb upb npb and upb has complete languages among other results we show that p up if and only if there exists a set s in p of boolean formulas with at most one satisfying assignment such that s sat is not in p p up coup if and only if there exists a set s in p of uniquely satisfiable boolean formulas such that no polynomial time machine can compute the solutions for the formulas in s if up has complete languages then there exists a set r in p of boolean formulas with at most one satisfying assignment so that sat r is complete for up finally we indicate the wide applicability of our techniques to counting and probabilistic classes by using them to examine the probabilistic class bpp there is a relativized world where bppa has no complete languages if bpp has complete languages then it has a complete language of the form b majority where b p and majority f f is true for at least half of all assignments is the canonical pp complete set

quantum and classical complexity classes separations collapses and closure properties we study the complexity of quantum complexity classes such as eqp bqp and nqp quantum analogs of p bpp and np respectively using classical complexity classes such as zpp wpp and c p the contributions of this paper are threefold first via oracle constructions we show that no relativizable proof technique can improve the best known classical upper bound for bqp bqp awpp journal of computer and system sciences 59 2 1999 240 to bqp wpp and the best known classical lower bound for eqp p eqp to zpp eqp second we prove that there are oracles a and b such that relative to a corp is immune to nqp and relative to b bqp is immune to pc p extending a result of de graaf and valiant technical report quant ph 0211179 quantum physics 2002 we construct a relativized world where eqp is immune to modpkp third motivated by the fact that counting classes e g lwpp awpp etc are the best known classical upper bounds on quantum complexity classes we study properties of these counting classes we prove that wpp is closed under polynomial time truth table reductions while we construct an oracle relative to which wpp is not closed under polynomial time turing reductions the latter result implies that proving the equality of the similar appearing classes lwpp and wpp would require nonrelativizable proof techniques we also prove that both awpp and app are closed under tup reductions we use closure properties of wpp and awpp to prove interesting consequences in terms of the complexity of the polynomial hierarchy of the following hypotheses nqp bqp and eqp nqp

computation times of np sets of different densities abstract in this paper we study the computational complexity of sets of different densities in np we show that the deterministic computation time for sets in np can depend on their density if and only if there is a collapse or partial collapse of the corresponding higher nondeterministic and deterministic time bounded complexity classes we show also that for np sets of different densities there exist complete sets of the corresponding density under polynomial time turing reductions finally we show that these results can be interpreted as results about the complexity of theorem proving and proof presentation in axiomatized mathematical systems this interpretation relates fundamental questions about the complexity of our intellectual tools to basic structural problems about p np conp and pspace discussed in this paper

pegasos primal estimated sub gradient solver for svm we describe and analyze a simple and effective stochastic sub gradient descent algorithm for solving the optimization problem cast by support vector machines svm we prove that the number of iterations required to obtain a solution of accuracy epsilon is tilde o 1 epsilon where each iteration operates on a single training example in contrast previous analyses of stochastic gradient descent methods for svms require omega 1 epsilon 2 iterations as in previously devised svm solvers the number of iterations also scales linearly with 1 where is the regularization parameter of svm for a linear kernel the total run time of our method is tilde o d lambda epsilon where d is a bound on the number of non zero features in each example since the run time does not depend directly on the size of the training set the resulting algorithm is especially suited for learning from large datasets our approach also extends to non linear kernels while working solely on the primal objective function though in this case the runtime does depend linearly on the training set size our algorithm is particularly well suited for large text classification problems where we demonstrate an order of magnitude speedup over previous svm learning methods

on the algorithmic implementation of multiclass kernel based vector machines in this paper we describe the algorithmic implementation of multiclass kernel based vector machines our starting point is a generalized notion of the margin to multiclass problems using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks our notion of margin yields a direct method for training multiclass predictors by using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size we describe an efficient fixed point algorithm for solving the reduced optimization problems and prove its convergence we then discuss technical details that yield significant running time improvements for large datasets finally we describe various experiments with our approach comparing it to previously studied kernel based methods our experiments indicate that for multiclass problems we attain state of the art accuracy

some connections between bounded query classes and non uniform complexity let a x be the characteristic function of a consider the function cka x1 xk a x1 a xk we show that if cka can be computed in polynomial time with fewer than k queries to some set x then a p poly a generalization of this result has applications to bounded query classes circuits and enumerability in particular we obtain the following 1 assuming 3p 3p there are functions computable using f n 1 queries to sat that are not computable using f n queries to sat for f n o log n 2 if cka restricted to length n inputs can be computed by an unbounded fanin oracle circuit of size s n and depth d n with k 1 queries to some set x then a can be computed with an unbounded fanin non oracle circuit of size no k s n and depth d n o 1 3 assuming that ph 4p 4p and e 1 sat is not 2ne enumerable

optimal series parallel trade offs for reducing a function to its own graph consider the problem of computing a function given only an oracle for its graph for this problem we present optimal trade offs between serial and parallel queries in particular we give a fuction for which parallel access to its own graph is exponentially more expensive than sequential access

online ranking by projecting we discuss the problem of ranking instances in our framework each instance is associated with a rank or a rating which is an integer in 1 to k our goal is to find a rank prediction rule that assigns each instance a rank that is as close as possible to the instance s true rank we discuss a group of closely related online algorithms analyze their performance in the mistake bound model and prove their correctness we describe two sets of experiments with synthetic data and with the eachmovie data set for collaborative filtering in the experiments we performed our algorithms outperform online algorithms for regression and classification applied to ranking

corrigendum to jordan homomorphisms of upper triangular matrix rings linear algebra appl 439 12 2013 4063 4069 abstract in this paper we will present a new proof of the main theorem in linear algebra appl 439 2013 4063 4069

the boolean hierarchy of np partitions we introduce the boolean hierarchy of k partitions over np for k 3 as a generalization of the boolean hierarchy of sets i e 2 partitions over np whereas the structure of the latter hierarchy is rather simple the structure of the boolean hierarchy of k partitions over np for k 3 turns out to be much more complicated we formulate the embedding conjecture which enables us to get a complete idea of this structure this conjecture is supported by several partial results

separating complexity classes with tally oracles abstract long and selman 1986 proved that for most familiar pairs of complexity classes separating the classes with a tally oracle is no easier than truly separating the classes refuting a claim in the literature we prove for the first time that for many familiar pairs of complexity classes separating the classes with a tally oracle is easier than truly separating the classes

proximity control in bundle methods for convex proximal bundle methods for minimizing a convex functionf generate a sequence x k by takingxk 1 to be the minimizer of hat f k x u k x x k 2 2 where hat f k is a sufficiently accurate polyhedral approximation tof andu k 0 the usual choice ofu k 1 may yield very slow convergence a technique is given for choosing u k adaptively that eliminates sensitivity to objective scaling some encouraging numerical experience is reported

on boolean lowness and boolean highness the concepts of lowness and highness originate from recursion theory and were introduced into the complexity theory by schoning sch85 informally a set is low high resp for a relativizable class k of languages if it does not add adds maximal resp power to k when used as an oracle in this paper we introduce the notions of boolean lowness and boolean highness informally a set is boolean low boolean high resp for a class k of languages if it does not add adds maximal resp power to k when combined with k by boolean operations we prove properties of boolean lowness and boolean highness which show a lot of similarities with the notions of lowness and highness using kadin s technique of hard strings see kad88 wag87 ck96 bco93 we show that the sets which are boolean low for the classes of the boolean hierarchy are low for the boolean closure of 2p furthermore we prove a result on boolean lowness which has as a corollary the best known result see bco93 in fact even a bit better on the connection of the collapses of the boolean hierarchy and the polynomial time hierarchy if bh np k then ph 2p k 1 np k

removing redundancy from a clause abstract this paper deals with the problem of removing redundant literals from a given clause we first consider condensing a weak type of redundancy elimination a clause is condensed if it does not subsume any proper subset of itself it is often useful and sometimes necessary to replace a non condensed clause c by a condensation i e by a condensed subset of c which is subsumed by c after studying the complexity of an existing clause condensing algorithm we present a more efficient algorithm and provide arguments for the optimality of the new method we prove that testing whether a given clause is condensed is co np complete and show that several problems related to clause condensing belong to complexity classes that are probably slightly harder than np we also consider a stronger version of redundancy elimination a clause c is strongly condensed iff it does not contain any proper subset c such that c logically implies c we show that the problem of testing whether a clause is strongly condensed is undecidable

approximable sets much structural work on np complete sets has exploited sat s d self reducibility in this paper we exploit the additional fact that sat is a d cylinder to show that np complete sets are p superterse unless p np in fact every set that is np hard under polynomial time no 1 tt reductions is p superterse unless p np in particular no p selective set is np hard under polynomial time no 1 tt reductions unless p np in addition no easily countable set is np hard under turing reductions unless p np self reducibility does not seem to suffice far our main result in a relativized world we construct a d self reducible set in np p that is polynomial time 2 tt reducible to a p selective set

locating p poly optimally in the extended low hierarchy the low hierarchy in np and the extended low hierarchy have turned out to be very useful in classifying many interesting language classes and almost all of them could be located optimally therein however until now the exact location of p poly remained open

efficient algorithms for membership in boolean hierarchies of regular languages this paper provides efficient algorithms that decide membership for classes of several boolean hierarchies for which efficiency or even decidability were previously not known we develop new forbidden chain characterizations for the single levels of these hierarchies and obtain the following results the classes of the boolean hierarchy over level 1 of the dot depth hierarchy are decidable in nl previously only the decidability was known the same remains true if predicates mod d for fixed d are allowed if modular predicates for arbitrary d are allowed then the classes of the boolean hierarchy over level 1 are decidable for the restricted case of a two letter alphabet the classes of the boolean hierarchy over level 2 of the straubing therien hierarchy are decidable in nl this is the first decidability result for this hierarchy the membership problems for all mentioned boolean hierarchy classes are logspace many one hard for nl the membership problems for quasi aperiodic languages and for d quasi aperiodic languages are logspace many one complete for pspace

bounded query classes polynomial time machines having restricted access to an np oracle are investigated restricted access means that the number of queries to the oracle is restricted or the way in which the queries are made is restricted e g queries made during truth table reductions very different kinds of such restrictions result in the same or comparable complexity classes in particular the class p text np o log n can be characterized in very different ways furthermore the boolean hierarchy is generalized in such a way that it is possible to characterize p text np and p text np o log n in terms of the generalization

graph minimal uncolorability is d p complete in their excellent paper c h papadimitriou and m yannakakis py asked whether the minimal 3 uncolorability problem is among other critical problems d p complete this paper gives an affirmative answer to the above question we show that minimal k uncolorability is d p complete for all fixed k geq 3 furthermore for k 3 the reduction can be modified by using sensitive gadgets to resolve the planar case bounded vertex degree case and their combination

kolmogorov complexity and degrees of tally sets we show that either epm tally epbtt tally or epm tally ep1 tt tally ep2 tt tally ep3 tt tally where erp tally denotes the class of sets which are equivalent to a tally set under rp reductions furthermore the question of whether or not emp tally ebttp tally is equivalent to the question of whether or not ne predicates can be solved in deterministic exponential time the proofs use the techniques of generalized kolmogorov complexity as corollaries to some of the main results we obtain new results about the kolmogorov complexity of sets in p

on bounded queries and approximation this paper investigates the computational complexity of approximating several np optimization problems using the number of queries to an np oracle as a complexity measure the results show a tradeoff between the closeness of the approximation and the number of queries required for an approximation factor k n log log k n n queries to an np oracle can be used to approximate the maximum clique size of a graph within a factor of k n however this approximation cannot be achieved using fewer than log log k n n c queries to any oracle unless pe np where c is a constant that does not depend on k these results hold for approximation factors k n geq 2 that belong to a class of functions which includes any integer constant function log n log a n and n 1 a similar results are obtained for graph coloring set cover and other np optimization problems

enumerative counting is hard abstract an n variable boolean formula may have anywhere from 0 to 2n satisfying assignments can a polynomial time machine given such a formula reduce this exponential number of possibilities to a small number of possibilities we call such a machine an enumerator and prove that if there is a good polynomial time enumerator for p i e one where for every boolean formula f the small set has at most o f 1 e numbers then p np p p and probabilistic polynomial time equals polynomial time furthermore we show that p polynomial time turing reduces to enumerating p

bounded queries approximations and the boolean hierarchy this paper investigates nondeterministic bounded query classes in relation to the complexity of np hard approximation problems and the boolean hierarchy nondeterministic bounded query classes turn out be rather suitable for describing the complexity of np hard approximation problems the results in this paper take advantage of this machine based model to prove that in many cases np approximation problems have the upward collapse property that is a reduction between np approximation problems of apparently different complexity at a lower level results in a similar reduction at a higher level for example if maxclique reduces to log n approximating maxclique using many one reductions then the traveling salesman problem tsp is equivalent to maxclique under many one reductions several upward collapse theorems are presented in this paper the proofs of these theorems rely heavily on the machinery provided by the nondeterministic bounded query classes in fact these results depend on a surprising connection between the boolean hierarchy and nondeterministic bounded query classes

the boolean hierarchy ii applications the boolean hierarchy i structural properties j cai et al siam j comput 17 1988 pp 1232 252 explores the structure of the boolean hierarchy the closure of np with respect to boolean

the boolean hierarchy and the polynomial hierarchy a closer connection we show that if the boolean hierarchy collapses to level k then the polynomial hierarchy collapses to bh 3 k where bh 3 k is the k th level of the boolean hierarchy over sigma p 2 this is an improvement over the known results which show that the polynomial hierarchy would collapse to p np np o log n this result is significant in two ways first the theorem says that a deeper collapse of the boolean hierarchy implies a deeper collapse of the polynomial hierarchy also this result points to some previously unexplored connections between the boolean and query hierarchies of delta p 2 and delta p 3 namely begin eqnarray bh k rm co bh k implies bh 3 k rm co bh 3 k jot p rm np k p rm np k 1 implies p rm np rm np k 1 p rm np rm np k 2 end eqnarray

new variants of bundle methods in this paper we describe a number of new variants of bundle methods for nonsmooth unconstrained and constrained convex optimization convex concave games and variational inequalities we outline the ideas underlying these methods and present rate of convergence estimates

relating equivalence and reducibility to sparse sets for various polynomial time reducibilities r this paper asks whether being r reducible to a sparse set is a broader notion than being r equivalent to a sparse set although distinguishing equivalence and reducibility to sparse sets for many one or 1 truth table reductions would imply that p ne np this paper shows that for k truth table reductions k geq 2 equivalence and reducibility to sparse sets provably differ though gavalda and watanabe have shown that for any polynomial time computable unbounded function f cdot some sets f n truth table reducible to sparse sets are not even turing equivalent to sparse sets this paper shows that extending their result to the 2 truth table case would provide a proof that p ne np additionally this paper studies the relative power of different notions of reducibility and proves that disjunctive and conjunctive truth table reductions to sparse sets are surprisingly powerful refuting a conjecture of ko

quantifying the amount of verboseness we study the fine structure of the classification of sets of natural numbers a according to the number of queries which are needed to compute the n fold characteristic function of a a complete characterization is obtained relating the question to finite combinatorics in order to obtain an explicit description we consider several interesting combinatorial problems

a version of the bundle idea for minimizing a nonsmooth function conceptual idea convergence analysis numerical results during recent years various proposals for the minimization of a nonsmooth functional have been made amongst these the bundle concept turned out to be an especially fruitful idea based on this concept a number of authors have developed codes that can successfully deal with nonsmooth problems the aim of the paper is to show that by adding some features of the trust region philosophy to the bundle concept the end result is a distinguished member of the bundle family with a more stable behaviour than some other bundle versions the reliability and efficiency of this code is demonstrated on the standard academic test examples and on some real life problems

jordan homomorphisms of upper triangular matrix rings abstract in this paper we investigate jordan homomorphisms of upper triangular matrix rings and give a sufficient condition under which they are necessarily homomorphisms or anti homomorphisms

trust region newton method for logistic regression large scale logistic regression arises in many applications such as document classification and natural language processing in this paper we apply a trust region newton method to maximize the log likelihood of the logistic regression model the proposed method uses only approximate newton steps in the beginning but achieves fast convergence in the end experiments show that it is faster than the commonly used quasi newton approach for logistic regression we also extend the proposed method to large scale l2 loss linear support vector machines svm

observations on complete sets between linear time and polynomial time there is a single set that is complete for a variety of nondeterministic time complexity classes with respect to related versions of m reducibility this observation immediately leads to transfer results for determinism versus nondeterminism solutions also an upward transfer of collapses of certain oracle hierarchies built analogously to the polynomial time or the linear time hierarchies can be shown by means of uniformly constructed sets that are complete for related levels of all these hierarchies a similar result holds for difference hierarchies over nondeterministic complexity classes finally we give an oracle set relative to which the nondeterministic classes coincide with the deterministic ones for several sets of time bounds and we prove that the strictness of the tape number hierarchy for deterministic linear time turing machines does not relativize

a second step towards complexity theoretic analogs of rice s theorem abstract rice s theorem states that every nontrivial language property of the recursively enumerable sets is undecidable borchert and stephan 1997 initiated the search for complexity theoretic analogs of rice s theorem in particular they proved that every nontrivial counting property of circuits is up hard and that a number of closely related problems are spp hard the present paper studies whether their up hardness result itself can be improved to spp hardness we show that their up hardness result cannot be strengthened to spp hardness unless unlikely complexity class containments hold nonetheless we prove that every p constructibly bi infinite counting property of circuits is spp hard we also raise their general lower bound from unambiguous nondeterminism to constant ambiguity nondeterminism

jordan homomorphisms of upper triangular matrix rings over a prime ring abstract the aim of the paper is to prove that under a mild assumption every jordan homomorphism from an upper triangular matrix ring over a unital ring onto another upper triangular matrix ring over a unital prime ring of characteristic not 2 is either a homomorphism or an anti homomorphism this result is an extension of a classical theorem of herstein

bi immunity results for cheatable sets abstract an oracle a is k cheatable if there is a polynomial time algorithm to determine the answers to 2k parallel queries to a from the answers to only k queries to some other oracle b it is known that 1 cheatable sets cannot be bi immune for p in contrast we construct 2 cheatable sets that are bi immune for arbitrary time complexity classes in addition for each k we construct a set that is k 1 cheatable but not k cheatable we show that this separation does not hold with bi immunity we show that if a recursive set a is bi immune for p then there exists a nontrivial 1 cheatable set that is polynomial time m reducible to a consequently if np contains a set that is bi immune for p then np contains a set that is not polynomial time turing equivalent to a self reducible set

discrete time local value iteration adaptive dynamic programming admissibility and termination analysis in this paper a novel local value iteration adaptive dynamic programming adp algorithm is developed to solve infinite horizon optimal control problems for discrete time nonlinear systems the focuses of this paper are to study admissibility properties and the termination criteria of discrete time local value iteration adp algorithms in the discrete time local value iteration adp algorithm the iterative value functions and the iterative control laws are both updated in a given subset of the state space in each iteration instead of the whole state space for the first time admissibility properties of iterative control laws are analyzed for the local value iteration adp algorithm new termination criteria are established which terminate the iterative local adp algorithm with an admissible approximate optimal control law finally simulation results are given to illustrate the performance of the developed algorithm

reasoning with power defaults this paper introduces power default reasoning pdr a framework for non monotonic reasoning based on the domain theoretic idea of modeling default rules with partial information in a higher order setting pdr lifts a non monotonic operator at the base syntactic level to a well behaved almost monotonic operator in the higher order space of the smyth power domain effectively a space of sets of models working in the model space allows us to prove the dichotomy theorem and the extension splitting theorem leading to a more well behaved logic and modulo the usual complexity conjectures a less complex logic than standard default logic specifically we prove that skeptical normal default inference is a problem complete for co np 3 in the boolean hierarchy for strict propositional logic and np 4 complete in general these results by changing the underlying semantics contrasts favorably with similar results of gottlob j logic comput 2 3 1992 397 425 who proves that standard skeptical default reasoning is 2p complete furthermore we show that the skeptical non monotonic consequence relation defined using our domain theoretic semantics obeys all of the laws for preferential consequence relations defined by kraus lehmann and magidor in particular we get the property of being able to reason by cases and the so called law of cautious monotony both of these laws fail for the standard propositional default logic of reiter artificial intelligence 13 1980 81 132 but hold in pdr as a consequence of the dichotomy theorem and the extension splitting theorem

on the structure of bounded queries to arbitrary np sets kadin siam j comput 17 1988 pp 1263 1282 showed that if the polynomial hierarchy ph has infinitely many levels then for all k text p operatorname sat k subset text p operatorname sat k 1 this paper extends kadin s technique and shows that a proper query hierarchy is not an exclusive property of np complete sets in fact for any a in text np widehat text low 3 if ph has infinitely many levels then text p a k subset text p a k 1 moreover for the case of parallel queries text p a k 1 is not even contained in text p operatorname sat k these same techniques can be used to explore some other questions about query hierarchies for example if there exists any a such that text p a 2 text p operatorname sat 1 then ph collapses to delta 3 p

efficient exact inference with loss augmented objective in structured learning structural support vector machine svm is an elegant approach for building complex and accurate models with structured outputs however its applicability relies on the availability of efficient inference algorithms the state of the art training algorithms repeatedly perform inference to compute a subgradient or to find the most violating configuration in this paper we propose an exact inference algorithm for maximizing nondecomposable objectives due to special type of a high order potential having a decomposable internal structure as an important application our method covers the loss augmented inference which enables the slack and margin scaling formulations of structural svm with a variety of dissimilarity measures e g hamming loss precision and recall f beta loss intersection over union and many other functions that can be efficiently computed from the contingency table we demonstrate the advantages of our approach in natural language parsing and sequence segmentation applications

ultraconservative online algorithms for multiclass problems in this paper we study a paradigm to generalize online classification algorithms for binary classification problems to multiclass problems the particular hypotheses we investigate maintain one prototype vector per class given an input instance a multiclass hypothesis computes a similarity score between each prototype and the input instance and sets the predicted label to be the index of the prototype achieving the highest similarity to design and analyze the learning algorithms in this paper we introduce the notion of ultraconservativeness ultraconservative algorithms are algorithms that update only the prototypes attaining similarity scores which are higher than the score of the correct label s prototype we start by describing a family of additive ultraconservative algorithms where each algorithm in the family updates its prototypes by finding a feasible solution for a set of linear constraints that depend on the instantaneous similarity scores we then discuss a specific online algorithm that seeks a set of prototypes which have a small norm the resulting algorithm which we term mira for margin infused relaxed algorithm is ultraconservative as well we derive mistake bounds for all the algorithms and provide further analysis of mira using a generalized notion of the margin for multiclass problems we discuss the form the algorithms take in the binary case and show that all the algorithms from the first family reduce to the perceptron algorithm while mira provides a new perceptron like algorithm with a margin dependent learning rate we then return to multiclass problems and describe an analogous multiplicative family of algorithms with corresponding mistake bounds we end the formal part by deriving and analyzing a multiclass version of li and long s romma algorithm we conclude with a discussion of experimental results that demonstrate the merits of our algorithms

lower bounds and the hardness of counting properties rice s theorem states that all nontrivial language properties of recursively enumerable sets are undecidable borchert and stephan math logic quart 46 4 2000 489 504 started the search for complexity theoretic analogs of rice s theorem and proved that every nontrivial counting property of boolean circuits is up hard hemaspaandra and rothe theoret comput sci 244 1 2 2000 205 217 improved the up hardness lower bound to upo 1 hardness the present paper raises the lower bound for nontrivial counting properties from upo 1 hardness to fewp hardness i e from constant ambiguity nondeterminism to polynomial ambiguity nondeterminism furthermore we prove that no relativizable technique can raise this lower bound to fewp 1 ttp hardness we also prove a rice style theorem for np namely that every nontrivial language property of np sets is np hard and for a broad class of leaf language classes we prove a sufficient condition for the natural analog of rice s theorem to hold