computational complexity of counting problems on 3-regular planar graphs A variety of counting problems on 3-regular planar graphs are considered in this paper. We give a sufficient condition which guarantees that the coefficients of a homogeneous polynomial can be uniquely determined by its values on a recurrence sequence. This result enables us to use the polynomial interpolation technique in high dimension to prove the #P-completeness of problems on graphs with special requirements. Using this method, we show that #3-Regular Bipartite Planar Vertex Covers is #P-complete. Furthermore, we use Valiant's Holant Theorem to construct a holographic reduction from it to #2,3-Regular Bipartite Planar Matchings, establishing the #P-completeness of the latter. Finally, we completely classify the problems #Planar Read-twice 3SAT with different ternary symmetric relations according to their computational complexity, by giving several more applications of holographic reduction in proving the #P-completeness of the corresponding counting problems. Counting complexity is a natural extension of the complexity of decision problems. It is an active area and has received much attention since Valiant [15] introduced the complexity class #P. Many #P-complete problems have been discovered. Typical examples can be found in such references as [1, 5, 7, 8, 11, 14] .. On the other hand, Valiant [16] proposed a new theory of the holographic reduction that allows for gadgets with many-to-many correspondence, in which the individual correspondences among the solution fragments can no longer be identified. Using this reduction, he succeeded in designing polynomial time algorithms for a number of counting problems.. However the whole class #P is less well understood, perhaps due to the complexity of combinatorial structures of solutions to the problems. An interesting topic in this area is to determine the complexity of the counting problems with various restrictions. Many planar counting problems, such as #Planar 3SAT and #Planar Clique Cover etc., are studied in [8] . Counting problems on planar bipartite graphs with bounded degree or regular graphs are investigated in [14] . In this paper, we consider counting problems on 3-regular planar graphs, that is, the counting version of Planar Read-twice 3SAT. Here, read-twice means that each variable occurs twice.. Vadhan [14] showed by polynomial interpolation that the problem counting vertex covers is #P-complete on planar bipartite graphs with maximum degree 4, and on k-regular graphs for all k ≥ 5. Greenhill [7] improved the latter to k = 3. We show that #3-Regular Bipartite Planar Vertex Covers (#3RBP-VC) is #P-complete (Theorem 9).. Vadhan [14] also showed that the counting of matchings is #P-complete for planar bipartite graphs with maximum degree 6. We use #2,3-Regular Bipartite Planar λ-Matchings (#2,3RBP-λ-Matchings) to denote the problem of calculating the summation of weights of matchings for 2,3-regular bipartite planar graphs, where the weight of a matching is λ to the power t, and t is the number of unmatched vertices of degree 3. Valiant [16] gave a holographic algorithm for the problem #X-Matchings. Therefore, as a special case of #X-Matchings, #2,3RBP-(−3)-Matchings is also polynomial time computable. The authors are very grateful to Valiant for suggesting the problem whether #2,3RBP-λ-Matchings is #P-complete for all rational numbers λ except for 0 and −3 to us. We show that #2,3RBP-λ-Matchings is #P-complete for almost all rational numbers λ except for finitely many ones (Theorem 10).. We also consider the #Planar Read-twice 3SAT problems, where each clause is a ternary symmetric relation in three literals. Different relations may lead to different computational complexity of the corresponding problems. We establish the #P-completeness for three of them, while together with the known cases, we characterize the computational complexity for this class of problem. For example, we show that #Planar Read-twice Monotone 3CNF is #P-complete, improving a result in [8] . A special interesting case is the problem #Planar Read-twice Not-equal-2 3SAT, which is #P-complete, but we prove that a closely related problem, #Read-twice Positive Not-equal-2 3SAT, is polynomial time computable.. Our methodology contribution is a generalization of the polynomial interpolation technique to high dimension, which leads to our first result about #VC. All other #P-completeness results are proved by combining polynomial interpolation and Valiant's holographic reduction.. In Section 2, we introduce the problems, the holographic reduction, the polynomial interpolation theorem in high dimension and some basic facts. In Section 3, we prove that #3RBP-VC is #P-complete. In Section 4, we prove that #2,3RBP-λ-Matchings is #P-complete for almost all rational numbers λ except for finitely many ones. In Section 5, we build the results about #Read-twice 3SAT.. Our terminology and notations are standard; readers are referred to Papadimitriou [12] . This paper is a full version of [18] , extended by similar results. The study of counting and its computational complexity is interesting and important. However, we only have a limited understanding of how the complexity of counting problems behaves in restricted cases. The results of this paper improve the situation somewhat, but there are still many open problems. We hope that the methods developed here are useful in obtaining more #P-completeness results for other restricted counting problems. The high-dimension polynomial interpolation is only used in Section 3, leading to a strong result. The results in other sections are not restricted to bipartite graphs. #3RBP-VC seems to be a good candidate for reductions. For example, consider #Pl-OR-{ f }-SAT: if f can simulate a ternary function which takes nonzero values on (0, 0, 0) and (1, 1, 1) , and value 0 on the other inputs, then this problem is #P-complete by a simple polynomial interpolation technique. But by a result of Cook [4] there are not many such ternary relations.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Digraph decompositions and monotonicity in digraph searching We consider graph searching games on directed graphs and corresponding digraph decompositions. In particular we show that two important variants of these games -underlying DAG-and Kelly-decompositions -are nonmonotone. Furthermore, we explore the limits of algorithmic applicability of digraph decompositions and show that various natural candidates for problems, which potentially could benefit from digraphs having small "directed width", remain NPcomplete even on almost acyclic graphs. The seminal work of Robertson and Seymour in their graph minor project has focused much attention on graph decompositions and associated measures of graph connectivity such as tree-or path-width. Aside from the interest in graph structure theory, these notions have also proved fruitful in the development of algorithms.. Intuitively, tree-width measures the similarity of a graph to a tree. Thus trees have tree-width one and graphs of small tree-width can be decomposed into parts with at most tree-width (plus one) vertices in a tree-like manner. Similarly to trees, tree-decompositions allow for recursive algorithms, whose running time is linear in the size of the underlying graph -but exponential in its width. Together with linear time parameterized algorithms for constructing tree-decompositions, this implies that a huge number of NP-complete problems become tractable on graph classes of bounded tree-width (see [8, 7] for a survey on tree-width).. Graph Searching Games. Closely related to tree-width (and path-width) are so called graph searching games. Graph searching games are played by two players, the searcher and the fugitive, that simultaneously place tokens on the vertices of a graph. Whereas the fugitive has only one token and is restricted to move along paths in the graph that are not occupied by a searcher, the searcher controls an arbitrary amount of tokens and is free to move them anywhere on the graph. The aim of the searcher is to capture the fugitive, i.e. to force him into a position where he is not able to move any more. The minimum number of tokens needed by the searcher to capture the fugitive defines a natural graph invariant.. Within this general framework, there exist a range of variants defined by different abilities for both players. In particular one distinguishes between the visible and invisible variant. In the visible case, the searchers can see the fugitive and can adapt their strategy accordingly. In the invisible case, the fugitive's position is hidden from the searcher. Concerning the abilities of the fugitive one distinguishes between the so called inert variant, where the fugitive is only allowed to move if a searcher is placed on his current position, and the dynamic variant, where the fugitive can move in any step of the play. Combining this yields four main variants of which only three will be considered in this paper: visible and dynamic (vis), invisible and inert (inert), and invisible and dynamic (invis). The forth variant, visible and inert has recently been studied by Richerby and Thilikos [24] .. An important concept in the theory of graph searching games is monotonicity. A game is monotone, if whenever k searchers can catch a fugitive on a graph they can do so without allowing the fugitive to re-occupy vertices. In general, restricting the searchers to monotone strategies may require additional searchers. LaPaugh [18] gave a first proof of monotonicity for a graph searching game. Since then, monotonicity has been intensely studied and a large number of monotonicity results have been established. See e.g. [18, 6, 9, 3, 12, 13, 19, 27] or the survey [2] and references therein.. The importance of monotonicity in the context of graph decompositions results from the observation that many decompositions, like tree-and path-decompositions, can be defined in terms of monotone winning strategies for the searcher. Monotonicity for a game is often established through duality theorems for the underlying decomposition. Strategies for the fugitive provide the dual notion for the existence of a decomposition and yield natural obstructions for graphs having small decompositions. For example, the notion of a bramble is a natural formalisation of a winning strategy for the fugitive and provides an important obstruction for small tree-width (see [11, 20] ).. Despite the considerable interest and the large number of results in this field, two cases have so far resisted any attempts to solve the monotonicity problem -the graph searching game with a visible, dynamic fugitive and the game with an invisible, inert fugitive, both played on digraphs. It is these games that are closely related to DAG-and Kelly-decompositions [4, 15] . In this paper,we solve the problems by showing that both games are non-monotone.. Digraph decompositions. In recent years, attempts have been made to generalise the notion of tree-decompositions and their algorithmic applications to directed graphs. Clearly, we can define the tree-width of a directed graph as the tree-width of the undirected graph we get by ignoring the direction of edges, a process which leads to some loss of information. This loss may be significant, if the algorithmic problems we are interested in are inherently directed. A good example is the problem of detecting Hamiltonian cycles. While we know that this can be solved easily on graphs with small treewidth, there are directed graphs with very simple connectivity structure which have large tree-width. Therefore, several proposals have been made to extend the notions of tree-decompositions and tree-width to directed graphs (see [23, 16, 3, 5, 25, 15] ). In particular, Reed [23] and Johnson, Robertson, Seymour, and Thomas [16] introduce the notion of directed tree-width and they show that Hamiltonicity can be solved for graphs of bounded directed tree-width in polynomial time.. Following this initial paper, several alternative definitions of directed graph decompositions have been proposed, with the aim of overcoming some shortcomings of the original definition. Berwanger, Dawar, Hunter and Kreutzer [4] and Obdržàlek [22] introduce the notion of DAG-width and Hunter and Kreutzer [15] introduce the notion of Kelly-width. All three proposals are supported by algorithmic applications and various equivalent characterisations in terms of obstructions, elimination orderings, and, in particular, variants of graph searching games on directed graphs. However, so far the algorithmic applications are restricted to few classes of problems, in particular the problem of finding disjoint paths, Hamiltonian-cycles and similar linkage problems, and certain problems in relation to combinatorial games (parity games) played on graphs motivated by the theory of computer-aided verification. Whereas the tree-width of undirected graphs has been employed to solve a huge number of problems on graphs of small tree-width, the algorithmic theory of directed graph decompositions is not nearly as rich.. It is an obvious question whether this is due to the fact that digraph decompositions are a relatively new field of research, where the fundamental machinery first needs to be developed, or whether this is due to a general limitation of this approach to algorithms on digraphs. In this paper we systematically explore the range of algorithmic applicability of digraph decompositions. For this, we look at typical NP-hard problems on graphs -as they can be found in [14] , for instance -and identify those that are "suitable" for this approach, where by "suitable" we mean that the problems should be NP-hard in general but tractable on acyclic digraphs. The reason for the latter is that all digraph decompositions proposed so far measure in some way the similarity of a graph to being acyclic. In particular, acyclic graphs have small width in all of these measures. Hence, if a problem is already hard on acyclic digraphs, there is no point in studying the effect of digraph decompositions on this problem. We then identify representatives for the various types of "suitable" problems and ask whether they can be solved in polynomial time on graphs of small directed tree-, Kelly-or DAG-width or of small directed path-width.. The results we present in Section 4 show that the border for algorithmic applicability of digraph decompositions is rather tight. Essentially, as far as classical graph theoretical problems are concerned, disjoint paths and Hamiltonian-cycles can be detected efficiently on graphs of small directed tree-width, but all other problems we considered such as Minimum Equivalent Subgraph, Feedback Vertex Set (FVS), Feedback Arc Set, Graph Grundy Numbering, and several others are NP-complete even on graphs with a very low global connectivity and thus very low directed path or tree-width.. Organisation. The paper is organised as follows. In Section 2 we briefly recall basic notions from graph and game theory needed later. In Section 3 we give a formal description of graph searching games and present the first main result of this paper, the non-monotonicity of the two types of games mentioned above. In Section 4 we explore the algorithmic boundaries of the digraph decompositions known so far by showing NP-completeness for a number of problems on digraphs with bounded "width". We conclude and state some open problems in Section 5.

O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


LOPASS: A Low-Power Architectural Synthesis System for FPGAs With Interconnect Estimation and Optimization Abstract-In this paper, we present a low-power architectural synthesis system (LOPASS) for field-programmable gate-array (FPGA) designs with interconnect power estimation and optimization. LOPASS includes three major components: 1) a flexible high-level power estimator for FPGAs considering the power consumption of various FPGA logic components and interconnects; 2) a simulated-annealing optimization engine that carries out resource selection and allocation, scheduling, functional unit binding, register binding, and interconnection estimation simultaneously to reduce power effectively; and 3) a -cofamily-based register binding algorithm and an efficient port assignment algorithm that reduce interconnections in the data path through multiplexer optimization. The experimental results show that LOPASS produces promising results on latency optimization compared to an academic high-level synthesis tool SPARK. Compared to an early commercial high-level synthesis tool, namely, Synopsys Behavioral Compiler, LOPASS is 61.6% better on power consumption and 10.6% better on clock period on average. Compared to a current commercial tool, namely, Impulse C, LOPASS is 31.1% better on power reduction with an 11.8% penalty on clock period.Index Terms-Behavioral synthesis, field-programmable gate array (FPGA), interconnect, power optimization.

O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I B B I


Incremental Construction of Minimal Deterministic Finite Cover Automata We present a fast incremental algorithm for constructing minimal DFCA for a given language. Since it was shown that the minimal DFCA for a language L has less states than the minimal DFA for the same language L, this technique seems to be the best choice for incrementally building the automaton for a large language, especially when the number of states in the DFCA is significantly less than the number of states in the corresponding minimal DFA. We have implemented the proposed algorithm and have tested it against the best known DFCA minimization technique. We have witnessed in recent years a growing interest in the design of incremental algorithms for finite automata [1, 7, 8, 11, [15] [16] [17] [18] [19] [20] . The reason behind the (renewed) interest for such incremental algorithms used for building a minimal Deterministic Finite Automata (DFA) for a given dictionary (finite language) comes from the observation that such an incremental algorithm could have much smaller memory requirements than a "global" minimization algorithm with little or no increase in the time complexity of the overall minimization process.. The small memory requirements for incremental algorithms (as opposed to "classical" minimization techniques) come from the fact that the DFA for a finite language is built word-by-word and minimized as words are inserted into the DFA. In this way the state complexity (and thus the memory requirements) of the incrementally built DFA could remain small as compared to the state complexity of a trie built for the whole dictionary as a first step and then minimizing the trie into a small DFA using a fast algorithm such as the one described by Hopcroft which requires O(n log(n)) time and O(n) space.. In the current paper we continue the research work in the incremental algorithms area, with the observation that for finite languages there is the recently defined concept of Deterministic Finite Cover Automata (DFCA) [6] . To conserve even more memory during the intermediate steps of the construction of the automaton we will devise an incremental algorithm for DFCA. We have proved in [4] that when transforming an NFA into a DFA and also into a DFCA the DFA can have exponentially more states than the DFCA; thus, there is a large class of languages for which the DFCA is the desirable representation as opposed to the DFA. For the recent results and properties of DFCA we refer the reader to [2] [3] [4] 6, 10, 12, 13] . We note that a Hopcroft-type algorithm (with O(n log(n)) time and O(n) space complexities) for the minimization of DFCA was described in [10] , but no incremental algorithms for DFCA are known. Another advantage of incremental solutions, beside efficiency, is the maintenance of the automaton, since the technique for increasing the number of words in the dictionary is already built-in. We fill this gap by describing an incremental algorithm for DFCA in the current paper. We have implemented in the Grail+ package [22] both the algorithm from [10] and the incremental algorithm proposed in the current paper; the preliminary tests suggest that the incremental algorithm is far superior with respect to the memory requirements as opposed to the Hopcroft-like algorithm, while no noticeable slow-down was observed for the languages tested.. The presented algorithm has complexity O(kn) in time and O(n 2 ) in space for adding a word of size k into a DFCA with n states. The time complexity is considered linear in literature in such a case (see [16] ) due to the fact that the size of a word from the language is usually much smaller than the size of the automaton accepting the language. Thus, we provide a fast incremental algorithm (having the same time complexity as the known incremental algorithms for DFA described in [20] ), but with a small increase in the memory requirements. This increase is small since, in practice, the complexity n is usually of logarithmical order of the complexity (trie size) of non-incremental algorithms. We will give an example of a language containing 2 n words for which the Hopcroft-like algorithm for DFCA requires O(2 n ) space, whereas our algorithm requires O(n 2 ) space.. For more information on incremental algorithms for DFA we refer the reader to [1, 7, 8, 11, [16] [17] [18] [19] [20] . It is worth noting that the paper [7] could be very interesting as it provides a comparison between the major algorithms (incremental/nonincremental) for the DFA, the comparison being performed using various dictionaries.

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O


Covering a laminar family by leaf to leaf links The Tree Augmentation Problem (TAP) is: given a tree T = (V, E) and a set E of edges (called links) on V disjoint to E, find a minimum size edge subset F ⊆ E so that T + F is 2-edge-connected. TAP is equivalent to the problem of finding a minimum size edge-cover F ⊆ E of a laminar set-family. We consider the restriction LL-TAP of TAP to instances when every link in E connects two leaves of T . The best approximation ratio for TAP is 3/2 [3, 4, 5], and no better ratio was known for LL-TAP. All the previous approximation algorithms that achieve a ratio better than 2 for TAP, or even for LL-TAP, were quite involved.For LL-TAP we obtain the following approximation ratios: 17/12 for general trees, 11/8 for trees of height 3, and 4/3 for trees of height 2. We also give a very simple 3/2-approximation algorithm (for general trees) and prove that it computes a solution of size at most min{ 3 2 t, 5 3 t * }, where t is the minimum size of an edge-cover of the leaves, and t * is the optimal value of the natural LP-relaxation for the problem of covering the leaf edges only. This provides the first evidence that the integrality gap of a natural LP-relaxation for LL-TAP is less than 2. We consider the following problem:. Tree Augmentation Problem (TAP) Instance: A tree T = (V, E) and a set E of edges (called links) on V disjoint to E. Objective: Find a min-size edge subset F ⊆ E so that T + F is 2-edge-connected.. TAP is equivalent to the problem of finding a minimum size edge-cover of a laminar family; namely, given a graph G = (V, E) and a laminar family E on V , we seek a minimum size edge-set F ⊆ E so that for every S ∈ E there exists uv ∈ F with u ∈ S and v / ∈ S. Laminar families play an important role in network design problems, c.f. [9] . See also surveys in [8, 11] for various network design problems and applications of laminar families in the analysis of algorithms for such problems.. Fredrickson and Jájá [6] showed that TAP is NP-hard even for trees of height 2, and gave a 2-approximation algorithm for the more general weighted version of TAP, when links have weights and we seek a minimum weight augmenting edge set F ⊆ E. Achieving a ratio better than 2 for (unweighted) TAP was posed as a major open problem in graph connectivity in the survey by Khuller [10] . This open question was resolved by Nagamochi [12] that gave a (1.875 + ε)-approximation scheme for TAP. The currently best approximation ratio known for TAP is 3/2, by Even, Feldman, Kortsarz, and Nutov: see the conference version in [3] , and the two part full version in [4, 5] .. For S ⊆ V let δ(S) denote the set of links in E with exactly one endnode in S; let E(S) denote the set of links in E with both endnodes in S. For E ⊆ E and x ∈ R E let. x(E ) = e∈E x e . Let τ * denote the optimal value of the following standard LP-relaxation for a TAP instance at hand:. s.t. x(δ(S)) ≥ 1 ∀S ∈ E x e ≥ 0 ∀e ∈ E Let τ denote the optimal value of a stronger LP-relaxation, which is obtained by adding to (1) the "leaf edge-cover integrality constraints" (see Chapter 27 in [13] ):. s.t. x(δ(S)) ≥ 1 ∀S ∈ E. x(E(U ) ∪ δ(U )) ≥ |U |/2 ∀U ⊆ L with |U | odd x e ≥ 0 ∀e ∈ E Let t * denote the optimal value of the relaxation of (1) for the problem of covering only the leaf edges, and similarly t is defined by relaxing (2) . Namely:. t = min {x(E) : x(E(U ) ∪ δ(U )) ≥ |U |/2 ∀U ⊆ L with |U | odd, x ≥ 0} .. Let LL-TAP be the restriction of TAP to instances when the endnodes of every link are leaves of T . In the covering laminar family setting, this is equivalent to requiring that every link has its endnodes in the minimal members of the laminar family, or that the laminar family contains all the singletons, see [7] . The case when T is a star is equivalent to the Edge-Cover problem. In this case τ * = t * , and τ = t is the optimal solution value, see. Chapter 27 in [13] . In general, τ ≥ τ * and t ≥ t * , and an equality may not hold.. LL-TAP was studied in several papers. Garg, Khandekar, and Talwar [7] gave a 5/3-approximation algorithm for LL-TAP. Cheriyan, Jordán, and Ravi [1] showed that even the special case of LL-TAP when the set of links forms a cycle on the leaves remains NP-hard, and gave, for this restricted case, a 4/3-approximation algorithm that computes a solution of size at most 4/3 · τ * . Motivated by this, they conjectured that the integrality gap of the LP-relaxation (1) for the general TAP is 4/3. This conjecture was recently disproved by Cheriyan, Karloff, Khandekar, and Könemann [2] , that showed that the integrality gap of (1) is at least 3/2. It is believed that the integrality gap of (1) is less than 2, but so far there was no evidence that this is so even for LL-TAP. We also note that so far, all the approximation algorithms that achieve a ratio better than 2 for TAP [12, 4, 5] , or even for LL-TAP [7, 1] , were quite involved.. Here is some notation used in the paper. Given an instance T = (V, E), E of TAP, for u, v ∈ V let (u, v) ∈ E denote the edge in T and uv the link in E between u and v. Fix a designated node r to be the root of T ; this defines a partial order on the nodes of T . For a, b ∈ V , a ≺ b means that a is a proper descendant of b, and if (a, b) ∈ E then b = p(a) is the parent of a. A node is a leaf of T if it has no descendants; let L = L(T ) the set of leaves of T . T uv denotes the unique uv-path in T , and T a denodes the subtree of T rooted at a. lca(u, v) is the least common ancestor of u, v in T . A link uv covers an edge e if e ∈ T uv . For a set L of leaves, up(L ) is a link in E with an endpoint in L covering the highest edge. For a link set E and a subtree T of T let T ∩ E be the set of links in E with both endnodes in T ; let V (E ) be the set of endnodes of the links in E . If we add a link uv to a partial solution F , then T uv belongs to the same 2-edge-connected component of T + F . Hence, we may contract T uv . Since all contractions we do are induced by subsets of links, we refer to the contraction of every 2-edge-connected component of T + F into a single node simply as the contraction of the links in F . Definition 1.1 A link uv ∈ T a ∩ E is redundant for a node a = r if every link that covers the edge (a, p(a)) is incident to u or to v, namely, if {x ∈ T a : there is xy ∈ E with y ∈ T − T a } ⊆ {u, v}.. Let R denote the set of redundant links. We will consider LL-TAP, when every link has both endnodes in L, and obtain the following results: Theorem 1.1 For LL-TAP, opt ≥ |L| − |M |, and there exists a polynomial time algorithm that computes a solution F of size |F | ≤ |L| − |M |/2 ≤ 3/2 · opt, where M is a maximum matching in E − R. Furthermore, if the LL-TAP instance has no redundant links, then 2 Algorithm with ratio 3/2 (Proof of Theorem 1.1)

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Sensitivity analysis and determination of free relaxation parameters for the weakly-compressible MRT-LBM schemes It ha been proved that the lattice Boltzmann Methods (LBMs) are very efficient for computational fluid dynamics purposes, and for capturing the dynamics of weak acoustic fluctuations. Especially, the multi-relaxation-time lattice Boltzmann method (MRT-LBM) appears as a very robust scheme with high precision. It is well-known that there exist several free relaxation parameters in the MRT-LBM. Although these parameters have been tuned via linear analysis, the sensitivity analysis of these parameters and other related parameters are still not sufficient for detecting the behaviors of the dispersion and dissipation relations of the MRT-LBM. Previous researches have shown that the bulk dissipation in the MRT-LBM induces a significant over-damping of acoustic disturbances. This indicates that MRT-LBM cannot be used to obtain the correct behavior of pressure fluctuations because of the fixed bulk relaxation parameter. In order to cure this problem, an effective algorithm has been proposed for recovering the linearized Navier-Stokes equations (L-NSE ) from the linearized MRT-LBM (L-MRT-LBM). The recovered L-NSE appear as in matrix form with arbitrary order of the truncation errors with respect to δt. Then, in wave-number space, the first/second-order sensitivity analyses of matrix eigenvalues are used to address the sensitivity of the wavenumber magnitudes to the dispersion-dissipation relations. By the first-order sensitivity analysis, the numerical behaviors of the group velocity of the MRT-LBM are first obtained. Afterwards, the distribution sensitivities of the matrix eigenvalues corresponding to the linearized form of the MRT-LBM are investigated in the complex plane. Based on the sensitivity analysis and the recovered L-NSE, we propose some simplified optimization strategies to determine the free relaxation parameters in the MRT-LBM. Meanwhile, the dispersion and dissipation relations of the optimal MRT-LBM are quantitatively compared with the exact dispersion and dissipation relations. At last, some numerical validations on classical acoustic benchmark problems are shown to assess the new optimal MRT-LBM. The lattice Boltzmann method (LBM) has emerged as an innovative mescoscopic numerical method for the computational modelling of a wide variety of complex fluid flows [1] . The early researches about the dispersion and dissipation relations of the classical LBM (BGK-LBM) show that the LBM are well suited to capture the weak acoustic pressure fluctuations [2, 3, 4, 5] . But, the BGK-LBM always suffers from an unstability drawback. In order to improve the stability of the BGK-LBM, the MRT-LBM was derived in moment space based on the generalized lattice Boltzmann equation [6, 7] . Thanks to its adjustable bulk viscosity, the MRT-LBM can guarantee a better stability by a high value of the bulk viscosity [3, 7] . In most researches and simulations, the relaxation parameters of the MRT-LBM were chosen according to the results of the linear analysis [6, 7] . Results of the linear von Neumann analysis of the. where the square matrices B (n) and B are functions of wave-number k and base-flow uniform velocity u, and B (n) can be regarded as a perturbation of B [8] . The vector W denotes the perturbed macroscopic fluid flow conservative quantities (such as, density ρ, momentum j, · · ·). Denoting M (n) = B (n) − B the perturbation matrix, the matrix M (n) will be the function of all relaxation parameters of the MRT-LBM for n > 2. With the aid of the matrix B (n) , we only consider the influence of the free parameters in the L-MRT-LBM on the hydrodynamic modes. This is of great benefit to determine the free relaxation parameters by our proposed optimization strategies. In order to establish the optimization strategies, the parameter sensitivities are investigated in the wave-number space. Then, the sensitivity properties of the free parameters on the hydrodynamic modes and the non-hydrodynamic modes are obtained. According to the sensitivity analysis, we propose a class of the simplified optimization strategies to minimize the dispersion and dissipation errors for the MRT-LBM in the wave-number space, based on the matrix perturbation theory and the theory of the modified equations. The proposed methodology leads to a relation between the recovered L-NSE and the exact L-NSE, and free parameters can be determined by this process. Then, the optimal dispersion and dissipation relations are investigated by von Neumann analysis. It is noted that this famous and reliable analysis method has been revisited and extended [12] .. It is necessary to point out that our proposed original methodology is more general than the usual method for recovering the higher-order lattice Boltzmann schemes [13, 14] , although we also use the Taylor expansion method. Our method appears as an easy-to-hand recursive algorithm in the wave-number space, which is completely different from the method proposed in [13] .. In next section, the fundamentals of the LBM are surveyed, and the algorithm used to establish the transformation relation from the L-MRT-LBM to the L-NSE is given. In the third section, with the aid of von Neumann method, the sensitivity properties of the free parameters to the dispersion and dissipation relations are analyzed in the wave-number space. Then, the optimization strategies are studied in section 4. In the last section, the optimal MRT-LBM schemes are validated considering some benchmark problems. In this paper, the researches are focused on the sensitivity analysis and the determination of the relaxation parameters in the MRT-LBM. By the investigations of the sensitivity, it is concluded that the numerical behaviors of the dispersion and dissipation are mainly dependent on the variation of the wave number k. From the behaviors of the eigenvalues of M mrt in the complex planes, it is observed that the distribution of the eigenvalues has the certain shapes if the wave number k is fixed and the velocity vector is regarded as a random variable. According to the sensitivity analysis, the integral spaces of the optimization strategies are simplified for 2D/3D problems. Then, by von Neumann analysis, the numerical performances of the optimization strategies are validated. Finally, by the benchmark acoustic problems and considering the ignorable small viscosity in the LBS, the optimal MRT-LBM is compared with the BGK-LBM and the used high-order compact/DRP schemes . The numerical results demonstrate that the optimal MRT-LBM can be used to simulate the acoustic problems very well. Meanwhile, the optimal MRT-LBM The transformation matrice for MRT-LBM with 9 discrete velocities is described by . For the weakly-compressible MRT-LBM, the matrix Ψ is given by . For the MRT-LBM similar to an incompressible LBS, the matrix Ψ is given by . Appendix A.2. For three dimensional MRT-LBM with 15 discrete velocities The transformation matrice for MRT-LBM with 15 discrete velocities is described by

B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O B O O O B O O O O O O O B O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O B O O O O O O B I O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O B O B O O O O B O O O O O O B O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O B O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O


Operative assessment of predicted generalization errors on non-stationary distributions in data-intensive applications Data-intensive applications use empirical methods to extract consistent information from huge samples. When applied to classification tasks, their aim is to optimize accuracy on unseen data hence a reliable prediction of the generalization error is of paramount importance. Theoretical models, such as Statistical Learning Theory, and empirical estimations, such as cross-validation, can both fit data-mining classification domains very well, provided some crucial assumptions are verified in advance. In particular, the stationary distribution of the observed data is critical, although it is sometimes overlooked in practice. The paper formulates an operative criterion to verify the stationary assumption; the method applies to both theoretical and practical predictions of generalization errors. The analysis addresses the specific case of clustering-based classifiers; the K-Winner Machine (KWM) model is used as a reference for its known theoretical bounds; cross-validation provides an empirical counterpart for practical comparison. The criterion, based on efficient unsupervised clustering-based probability distribution estimation, is tested experimentally on a set of different, data-intensive applications, including: intrusion detection for computer-network security, optical character recognition, text mining and pedestrian detection. Experimental results confirm the effectiveness of the proposed approach to efficiently detect non stationarity. In data-intensive applications, clustering methods arrange huge amounts of data into a structured representation and search for relevant information [1] [2] . The vast datasets and the heterogeneous descriptions of patterns set stringent requirements on the algorithms adopted; when empirical classifiers aim to optimize prediction on unseen data [1] , attaining an accurate estimate of the run-time generalization error is a critical issue. Several methods in the literature have tackled that problem from both a practical [3] and a theoretical viewpoint [4] [5] [6] .. From a practical viewpoint, empirical estimates such as cross-validation methods often support the prediction of the run-time generalization performance in real applications [30] . The literature reports that these techniques are quite accurate and outperform theoretical models in complex classifier design [29] . From a theoretical viewpoint, within the framework of Statistical Learning Theory [26] , the formulation based on the Vapnik-Chervonenkis dimension (d VC ) [7] exhibits a theoretical foundation: given a classifier C, and its associated class of decision functions, the d VC is the largest number of patterns that C can correctly classify; this makes d VC a reliable measure of complexity of a classification algorithm. Unfortunately, the resulting bounds to the generalization error often prove impractical for a variety of reasons. First, Vapnik's theory stems from a worstcase analysis, hence it usually requires a huge number of patterns to tighten generalization bounds down to reasonable ranges. Secondly, many practical classifiers are so powerful that the crucial parameter, d VC , measuring a model's complexity grows uncontrollably (e.g. SVM with Guassian kernel). Finally, few families of clustering-based classifiers have been fully characterized in compliance with Vapnik's theory. This is, incidentally, the case of the K-Winner Machine (KWM) model [8] , which is used here as the clustering-based framework for pattern classification.. In spite of the above limitations, Statistical Learning Theory can yet be of practical significance when dealing with clustering and data-mining [1] , since data mining applications are typically rich in patterns and can therefore offer the required large samples. Moreover, the complexity of clustering-based classifiers often proves much lower than that of other approaches [1] .. A crucial prerequisite in applying both theoretical and empirical predictions, however, is that the probability distribution of data is stationary [26] ; such a condition holds in many practical testbeds and is often assumed to hold implicitly. In some data-intensive domains, however, the stationary nature of data distributions may prove questionable, either because the data refer to a phenomenon whose time-varying nature is overlooked, or because the original sample is so large that new samples stem from unexplored areas of the probability distribution, hence test data are virtually uncorrelated from training ones. Retraining (either from scratch or as an update of existing learning results) is a typical solution to that problem, but in data-intensive applications it may prove very expensive. This may occur for a variety of reasons: for instance, because the actual process for updating training results is difficult to design or implement, or because the amount of data is excessive, or because older data might not be easily accessible.. In general, non stationarity can be classified according to two different categories: covariate shift [38] refers to those cases in which the non stationarity only affects the pattern probability distribution P(x); concept drift [40] refers to those case in which non stationarity is confined to the target probability distribution P(c|x). This paper addresses covariate shift and tackles the stationarysampling issue from the conventional viewpoint of validation methods for classifier training [26] ; the basic observation is that non-stationary distributions ultimately give rise to discrepancies between the data distributions in the training and test phases. To this aim, an efficient and reliable method to estimate the pdf and assess the stationarity of input data is required. The proposed criterion asses the non-stationarity by casting the original multivariate problem to an univariate problem via a clustering procedure. This procedure avoids the curse of dimensionality and the possible numerical instabilities that can occur using traditional parametric methods such as Parzen Windows or Mixture of Gaussian Models [31] [32] [33] [34] . Indeed, the class of problems covered by the proposed methodology is not strictly limited to covariate shift, as the procedure can effectively apply also to problems in which non stationarity affects both the pattern probability distribution and the target probability distribution.. The approach proposed in this paper adopts the KWM model [8] as classifier. The rationale behind such choice is twofold. KWM yields tight bounds to generalization performance [8] and inherently supports multi-class classification tasks [9] . These features make KWM profitably suitable for data-intensive applications and for evaluating the applicability of Vapnik's generalization predictions accordingly, together with conventional cross-validation methods.. The proposed methodology improves and generalizes the preliminary analysis discussed in [35] . Experiments first show the approach validity in a synthetic domain, mainly to provide an intuitive demonstration of the basic non stationarity detection principle; the method is then tested in a group of complex real-world problems: the detection of intrusions in computer networks, Optical. Character Recognition for numerical patterns, Emails Spam detection and Pedestrian Detection. The "KDD Cup 1999" dataset [10] , the Manuscript NIST (MNIST) OCR dataset [24] , the Spam Assassin dataset [37] and the Daimler dataset [39] provided the related experimental domains.. Experimental results show that the proposed criterion successfully detected the nonstationary/stationary nature of the proposed domains. Furthermore, an additional experiment it is reported to point out the differences between a problem characterized by covariate shift and a problem characterized by concept drift. Such experiment shows that the unsupervised nature of the proposed method cannot be applied to such kind of non stationarity where the drift is only limited to the target values.. The paper is organized as follows. Section 2 briefly illustrates error predictions methods and the K-Winner Machine model. Section 3, the core section, introduces the criterion designed to validate the applicability of generalization error estimation by stationarity assessment. Section 4 presents the experimental results obtained on reference, real-word testbeds, and deals with peculiar theoretical and practical emerging issues. Some concluding remarks are made in Section 5. The baseline of the presented research is that non stationarity detection is a notable practical problem, especially in data mining problems where a huge amount of samples are provided. Appendix. The K-Winner-Machine training algorithm. The following pseudocode sketches the KWM training algorithm, as proposed in [8] . The conventions are repeated for the reader's convenience:. is the set of N c possible pattern classes;. • A real-valued vector (x l , c(x l ) ) denotes a labeled pattern drawn from the space ℜ. is a set of N h labeled prototypes;. is the prototype that represents a pattern, x;. • P n is the data-space partition that is covered by the n-th prototype according to a minimumdistance criterion; it coincides with the Voronoi region associated with w n ;. is the share of patterns that lie in P n and belong to the h-th class; as a consequence of this definition, one has that, for each n-th partition: For the sake of completeness and clarity, in the following the actions executed by the proposed procedure in two opposite scenarios are analyzed:. 1) Stationary Case. If T and S are drawn from the same distribution P(x), then T and S alone can be used to estimate the underlying P(x). In other words, the discrete probability. are both reliable estimates of P(x) (under the assumption of a data-intensive problem). Thus, when applying step 4 of the proposed procedure (see pseudo-code above), one obtains a pair of sets (a training set, T J , and a test set, S J ) that leads to a consistent estimates of P(x) as well. As a consequence,. 2) Non Stationary Case. In this case T and S are drawn from two distinct distributions . . Such discrepancy is caused by the fact that ). In the above scenarios the merge-shuffle and split operation has no effect on distributions in the stationary case; instead in the non stationary case the merge-shuffle and split operation induces a change in the discrepancy values; in both cases. is the adaptive threshold that allows to discriminate between stationarity or not.. If stationarity is verified, the theoretical assumptions underlying Statistical Learning Theory hold, and the bound formulation (#1) or cross-validation (#3) are valid. Otherwise, when ( ). , one might infer that the original sampling process was not stationary, hence a direct application of theoretical results (#1) or empirical estimates (#3) is questionable. Clearly the artificial dataset ( J S , J T ) is built whenever a stationary 'reference' dataset is not provided. Indeed, the proposed procedure to validate the applicability of generalization-theory bounds can eventually exploits, when needed, the implementation of a resampling strategy, which can of course be repeated several times to enhance the statistical robustness of numerical estimates.. Two aspects make the methodology presented above suitable for data-intensive application: • in a single step one can obtain both the classification of data and the reliable estimation of the data distribution.. • the obtained estimation is based upon an univariate pdf, thus avoiding the usual issues that arises when classical methods such as Parzen Windows [32] and Mixture of Gaussians [34] when high dimensional spaces are involved.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The Rate of Convergence of AdaBoost The AdaBoost algorithm was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the "exponential loss." Unlike previous work, our proofs do not require a weak-learning assumption, nor do they require that minimizers of the exponential loss are finite. Our first result shows that at iteration t, the exponential loss of AdaBoost's computed parameter vector will be at most ε more than that of any parameter vector of ℓ 1 -norm bounded by B in a number of rounds that is at most a polynomial in B and 1/ε. We also provide lower bounds showing that a polynomial dependence on these parameters is necessary. Our second result is that within C/ε iterations, AdaBoost achieves a value of the exponential loss that is at most ε more than the best possible value, where C depends on the dataset. We show that this dependence of the rate on ε is optimal up to constant factors, i.e., at least Ω(1/ε) rounds are necessary to achieve within ε of the optimal exponential loss. The AdaBoost algorithm of Freund and Schapire (1997) was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. Despite extensive theoretical and empirical study, basic properties of AdaBoost's convergence are not fully understood. In this work, we focus on one of those properties, namely, to find convergence rates that hold in the absence of any simplifying assumptions. Such assumptions, relied upon in much of the preceding work, make it easier to prove a fast convergence rate for AdaBoost, but often do not hold in the cases where AdaBoost is commonly applied.. AdaBoost can be viewed as a coordinate descent (or functional gradient descent) algorithm that iteratively minimizes an objective function L : R n → R called the exponential loss (Breiman, 1999; Frean and Downs, 1998; Friedman et al., 2000; Friedman, 2001; Mason et al., 2000; Onoda et al., 1998; Rätsch et al., 2001; Schapire and Singer, 1999) . Given m labeled training examples (x 1 , y 1 ), . . . , (x m , y m ), where the x i 's are in some domain X and y i ∈ {−1, +1}, and a finite (but typically very large) space of weak hypotheses H = { 1 , . . . , N }, where each j : X → {−1, +1}, the exponential loss is defined as. where λ = λ 1 , . . . , λ N is a vector of weights or parameters. In each iteration, a coordinate descent algorithm moves some distance along some coordinate direction λ j . For AdaBoost, the coordinate directions correspond to the individual weak hypotheses. Thus, on each round, AdaBoost chooses some weak hypothesis and step length, and adds these to the current weighted combination of weak hypotheses, which is equivalent to updating a single weight. The direction and step length are so chosen that the resulting vector λ t in iteration t yields a lower value of the exponential loss than in the previous iteration, L(λ t ) < L(λ t−1 ). This repeats until it reaches a minimizer if one exists. It was shown by Collins et al. (2002) , and later by Zhang and Yu (2005) , that AdaBoost asymptotically converges to the minimum possible exponential loss. That is,. However, that work did not address a convergence rate to the minimizer of the exponential loss.. Our work specifically addresses a recent conjecture of Schapire (2010) stating that there exists a positive constant c and a polynomial poly() such that for all training sets and all finite sets of weak hypotheses, and for all B > 0,. In other words, the exponential loss of AdaBoost will be at most ε more than that of any other parameter vector λ of ℓ 1 -norm bounded by B in a number of rounds that is bounded by a polynomial in log N , m, B and 1/ε. (We require log N rather than N since the number of weak hypotheses will typically be extremely large.) Along with an upper bound that is polynomial in these parameters, we also provide lower bound constructions showing some polynomial dependence on B and 1/ε is necessary. Without any additional assumptions on the exponential loss L, and without altering AdaBoost's minimization algorithm for L, the best known convergence rate of AdaBoost prior to this work that we are aware of is that of Bickel et al. (2006) who prove a bound on the rate of the form O(1/ √ log t).. We provide also a convergence rate of AdaBoost to the minimum value of the exponential loss. Namely, within C/ǫ iterations, AdaBoost achieves a value of the exponential loss that is at most ǫ more than the best possible value, where C depends on the dataset. This convergence rate is different from the one discussed above in that it has better dependence on ǫ (in fact the dependence is optimal, as we show), and does not depend on the best solution within a ball of size B. However, this second convergence rate cannot be used to prove (1) since in certain worst case situations, we show the constant C may be larger than 2 m (although usually it will be much smaller).. Within the proof of the second convergence rate, we provide a lemma (called the decomposition lemma) that shows that the training set can be split into two sets of examples: the "finite margin set," and the "zero loss set." Examples in the finite margin set always make a positive contribution to the exponential loss, and they never lie too far from the decision boundary. Examples in the zero loss set do not have these properties. If we consider the exponential loss where the sum is only over the finite margin set (rather than over all training examples), it is minimized by a finite λ. The fact that the training set can be decomposed into these two classes is the key step in proving the second convergence rate.. This problem of determining the rate of convergence is relevant in the proof of the consistency of AdaBoost given by Bartlett and Traskin (2007) , where it has a direct impact on the rate at which AdaBoost converges to the Bayes optimal classifier (under suitable assumptions). It may also be relevant to practitioners who wish to have a guarantee on the exponential loss value at iteration t (although, in general, minimization of the exponential loss need not be perfectly correlated with test accuracy).. There have been several works that make additional assumptions on the exponential loss in order to attain a better bound on the rate, but those assumptions are not true in general, and cases are known where each of these assumptions are violated. For instance, better bounds are proved by Rätsch et al. (2002) using results from Luo and Tseng (1992) , but these appear to require that the exponential loss be minimized by a finite λ, and also depend on quantities that are not easily measured. There are many cases where L does not have a finite minimizer; in fact, one such case is provided by Schapire (2010) . Shalev-Shwartz and Singer (2008) have proven bounds for a variant of AdaBoost. Zhang and Yu (2005) also have given rates of convergence, but their technique requires a bound on the change in the size of λ t at each iteration that does not necessarily hold for AdaBoost. Many classic results are known on the convergence of iterative algorithms generally (see for instance Luenberger and Ye, 2008; Boyd and Vandenberghe, 2004) ; however, these typically start by assuming that the minimum is attained at some finite point in the (usually compact) space of interest, assumptions that do not generally hold in our setting. When the weak learning assumption holds, there is a parameter γ > 0 that governs the improvement of the exponential loss at each iteration. Freund and Schapire (1997) and Schapire and Singer (1999) showed that the exponential loss is at most e −2tγ 2 after t rounds, so AdaBoost rapidly converges to the minimum possible loss under this assumption.. In Section 2 we summarize the coordinate descent view of AdaBoost. Section 3 contains the proof of the conjecture, with associated lower bounds proved in Section 3.3. Section 4 provides the C/ǫ convergence rate. The proof of the decomposition lemma is given in Section 4.2.. Given: (x 1 , y 1 ), . . . , (x m , y m ) where x i ∈ X , y i ∈ {−1, +1} set H = { 1 , . . . , N } of weak hypotheses j : X → {−1, +1}. Initialize: D 1 (i) = 1/m for i = 1, . . . , m. For t = 1, . . . , T :. • Train weak learner using distribution D t ; that is, find weak hypothesis h t ∈ H whose correlation r t. where Z t is a normalization factor (chosen so that D t+1 will be a distribution).. Output the final hypothesis: In this paper we studied the convergence rate of AdaBoost with respect to the exponential loss. We showed upper and lower bounds for convergence rates to both an arbitrary target loss achieved by some finite combination of the weak hypotheses, as well as to the infimum loss which may not be realizable. For the first convergence rate, we showed a strong relationship exists between the size of the minimum vector achieving a target loss and the number of rounds of coordinate descent required to achieve that loss. In particular, we showed that a polynomial dependence of the rate on the ℓ 1 -norm B of the minimum size solution is absolutely necessary, and that a poly(B, 1/ε) upper bound holds, where ε is the accuracy parameter. The actual rate we derive has rather large exponents, and we discuss a minor variant of AdaBoost that achieves a much tighter and near optimal rate. For the second kind of convergence, using entirely separate techniques, we derived a C/ε upper bound, and showed that this is tight up to constant factors. In the process, we showed a certain decomposition lemma that might be of independent interest. We also study the constants and show how they depend on certain intrinsic parameters related to the singular values of the feature matrix. We estimate the worst case values of these parameters, and considering feature matrices with only {−1, 0, +1} entries, this leads to a bound on the rate constant C that is doubly exponential in the number of training examples. Since this is rather large, we also include bounds polynomial in both the number of training examples and the accuracy parameter ε, although the dependence on ε in these bounds is non-optimal.. Finally, for each kind of convergence, we conjecture tighter bounds that are not known to hold presently. A table containing a summary of the results in this paper is included in Figure 5 .. Since (1 + a) b ≥ 1 + ba for a, b non-negative, (9) will imply (u t /u t+1 ). t , which will complete our proof. To show (9), we first rearrange the condition on u t , u t+1 to obtain. Applying the fact (1 + c 0 u. Since c 1 ≥ 0, we may raise both sides of the above inequality to the power of c 1 to show (9), finishing our proof.

O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O B O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Formal verification of hybrid systems In formal verification, a designer first constructs a model, with mathematically precise semantics, of the system under design, and performs extensive analysis with respect to correctness requirements. The appropriate mathematical model for embedded control systems is hybrid systems that combines the traditional state-machine based models for discrete control with classical differential-equations based models for continuously evolving physical activities. In this article, we briefly review selected existing approaches to formal verification of hybrid systems, along with directions for future research. Categories and Subject DescriptorsVerification.Hybrid systems, Control systems, Model checking, Formal methodsModel-based design offers a promising approach for detecting and correcting errors in early stages of system design [33, 37, 49] . In this methodology, a designer first constructs a model, with mathematically precise semantics, of the system under design, and performs extensive analysis with respect to correctness requirements before generating the implementation from the model. Embedded systems, such as controllers in automotive, medical, and avionic systems, consist of a collection of interacting software modules reacting to a continuously evolving environment. The appropriate Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. EMSOFT'11, October 9-14, 2011, Taipei, Taiwan. Copyright 2011 mathematical model for design of embedded control systems is hybrid systems that combines the traditional models for discrete behavior with classical differential-and algebraicequations based models for dynamical systems. Such models can capture both the controller -the system under design, and the plant -the environment with continuously evolving physical activities in which the system operates. Given that (1) automated verification tools have recently been successful in finding bugs in "real-world" hardware protocols and device drivers [13, 15] , (2) tools such as Stateflow/Simulink are commonly used in automotive and avionics industry for modeling, and (3) high assurance is a necessity in safetycritical applications that deploy embedded software, formal verification of hybrid systems has been a vibrant research area for the past 20 years. This article is an overview of current research directions, aimed at providing an introductory "roadmap" rather than a comprehensive survey.In early 1990s, formal models for discrete reactive systems were integrated with models for dynamical systems [2, 41] . The model of hybrid automata [1, 2, 29] has emerged to be a popular choice. A hybrid automaton is an extended finitestate machine whose state consists of a mode that ranges over finitely many discrete values and a finite set of realvalued variables. Each mode is annotated with constraints that specify the continuous evolution of the system, and edges between modes are annotated with guards and updates that specify discrete transitions. For example, the behavior of a self-regulating thermostat can be described by a hybrid automaton with two modes, on and off, and a single real-valued variable T modeling the temperature. To specify how the temperature changes in the mode on, we can annotate the mode with a differential equation, say, T = k(70 − T ), for a constant parameter k. Alternatively, we can use a differential inequality, say, k 1 ≤Ṫ ≤ k2, for two constants k 1 and k2, to specify the dynamics approximately using only bounds on the derivative. Associating the mode on with an invariant constraint T ≤ 70 specifies that the system must exit this mode before the temperature exceeds 70. An edge from the mode on to the mode off with the guard T ≥ 68 specifies that, whenever the temperature exceeds 68, the system can discretely change its state by updating the mode to off. A hybrid automaton can naturally be interpreted as an infinite-state transition system, and this forms the basis for formalizing classical notions such 273 as safety verification, property-preserving abstractions, and simulation relations, for hybrid systems. The commercial modeling tools such as Stateflow/Simulink (see www.mathworks.com) and Modelica (see www.modelica. org) are routinely used in a wide range of industries. Conceptually, it should be possible to compile models expressed in such tools into formal notations such as hybrid automata. This has turned out to be difficult in practice due to the richness of features in commercial tools and the lack of a standardized formal semantics of such features (see [35] for efforts aimed at semantics-preserving transformations across modeling notations).In the safety verification problem for hybrid systems, we are given a hybrid systems model M , a set I of initial states of M , and a set S of "safe" states of M , and we want to check whether every execution of M starting in an initial state always stays within the set of safe states, and if not, report a violating execution as a counter-example. For instance, given the hybrid systems model of a collision avoidance protocol, we want to check whether the distance between two vehicles stays greater than the safety threshold for given initial conditions. Let us first note that the safety verification problem of hybrid systems cannot be solved algorithmically: with the exception of classes that severely restrict the allowed dynamics of real-valued variables such as timed automata [5] and initialized rectangular automata [32] , the safety verification problem is undecidable [1, 32] . Symbolic reachability algorithms for safety verification try to compute the set R of reachable states of a hybrid system M in an iterative manner starting from the set I of initial states. The algorithm checks, at every step of the iteration, if the current set R of reachable states is a subset of the set S of safe states, and if not, it terminates with a counter-example. In general, there is no termination guarantee, as the algorithm may keep adding more and more states to R without being able to deduce that the system is safe. The key challenge to efficient implementation is to identify a suitable representation for the set of states that supports the operations used by the iterative reachability computation.The tool HyTech was the first model checker to implement symbolic reachability analysis for hybrid systems [7, 31] . For a hybrid automaton with n real-valued variables, each reachable set is represented by associating a finite union of ndimensional polyhedra with each mode, where a polyhedron is represented as a conjunction of linear inequalities over variables. Such a polyhedra-based representation is appealing due to the use of polyhedra in many computing applications and the availability of open-source libraries for manipulating them (c.f. [12] ). The models are restricted to the class of linear hybrid automata (LHA): guards, updates, and invariants involve only linear expressions, and the dynamics is specified using differential inequalities that are linear constraints over first-order derivatives. For example, the LHA-admissible dynamicsẋ =ẏ ∧ 1 ≤ẋ ≤ 2 describes two-dimensional motion along the diagonal line with bounds on speed. For LHA, the polyhedral representation is closed under both discrete transitions corresponding to mode-switches and continuous evolution according to differential constraints in a mode: given a polyhedron R describing the set of current states, the set R of states that the system can reach after a discrete mode-switch to a mode m, is a polyhedron that can be computed effectively from R, and the set R of states that the system can reach as a result letting it evolve continuously according to the dynamics associated with the mode m, is also a polyhedron that can be computed effectively from R .The most commonly used dynamics in mathematical design of control systems involves linear differential equations: if x represents the vector of state variables, u represents the vector of input variables, a linear system is described by the equationẋ = Ax + Bu, where A and B are matrices of appropriate dimensions. Such dynamics is not allowed in LHA. Linear hybrid systems (LHS) refers to the class of hybrid automata where guards, updates, and invariants involve only linear expressions, and the dynamics is specified using linear differential equations. First note that the polyhedral representation is not closed under continuous evolution specified by linear differential equations: if x = 1 is the initial state and the dynamics is given by the linear differential equatioṅ x = x, the set of reachable states is the exponential curve given by e t , for real numbers t ≥ 0. Since manipulating transcendental functions is computationally difficult, a popular strategy, first advocated by the tool Checkmate [14], and later refined by the tool d/dt [11] , is to compute overapproximations of reachable sets using polyhedral representations. Given a polyhedron R representing the set of current states, to compute the set R of states that are reachable within a fixed time horizon Δ according to a given linear differential equation, the algorithm implements the following strategy. For every corner vertex v of R, it first computes the state v of the system at time Δ starting in state v. Then it computes the convex hull R 1 of all the corner vertices v of R and their respective images v . We are guaranteed that all the system trajectories start in the polyhedron R 1 and end in R 1 at time Δ, but are not necessarily inside R1 at intermediate times. The final step involves computing the polyhedron R 2 obtained by "face-lifting" R1: the number and normal vectors for facets of R 2 coincide with R1, but the facets are shifted outwards so as to include all reachable states upto time Δ. All these steps can be efficiently implemented for linear systems, and the resulting polyhedron R 2 is guaranteed to be a superset of the desired set R . The process can be repeated for successive time intervals, and the resulting approximation of the reachable set is called the flowpipe approximation.The complexity of operations on polyhedra is exponential in the number of dimensions (that is, the number of real-valued variables of the hybrid system), and since such operations are invoked repeatedly in symbolic analysis based on polyhedral representation, a significant body of work has been 274 aimed at battling this complexity and/or replacing polyhedra with alternative representations [11, 21, 36, 42] . Representation using zonotopes and support functions [22, 26] has so far resulted in the most scalable approach for the analysis of linear hybrid systems, leading to the tool SpaceEx that is able to analyze, for instance, a complex 28-dimensional helicopter controller [20] .In deductive verification, a designer interacts with a mechanized theorem prover to generate proofs of correctness of systems. For safety verification of discrete systems, a classical proof principle relies on the notion of inductive invariants: to show that all executions of a system M starting in an initial set I stay within a safe set S, we identify a state property ϕ such that (1) all initial states satisfy ϕ; (2) ϕ is a subset of the desired safety property S; and (3) the property ϕ is preserved locally across system transitions (that is, no transition of the system changes the value of ϕ from 1 to 0). In interactive verification, the user proposes a property ϕ, and the analysis tool checks if ϕ is an inductive invariant.The concept of inductive invariants for discrete systems has been generalized and adopted to continuous-time dynamical (and hybrid) systems. We will informally explain the first such notion, called barrier certificates [47, 48] . To show that a dynamical system M with dynamicsẋ = f (x) with initial set I satisfies a safety property S, we identify a function ψ from the states to reals such that (1) in all initial states, the value of ψ is nonnegative; (2) in all unsafe states (that is, states not in the safe set S), the value of ψ is negative; and (3) the Lie derivate of ψ with respect to the vector field f is positive on the boundary set (called the barrier) characterized by ψ(x) = 0. The first two conditions ensure that the barrier separates the initial states from the unsafe states, and the third condition ensures that system trajectories cannot escape the barrier from inside as at the barrier the flow field points inwards. Together, these conditions imply that ψ(x) ≥ 0 is an inductive invariant of the system. Typically, the desired function ψ is a polynomial function of the system variables. It is also worth noting that the barrier certificates are closely related to the notion of Lyapunov certificates for stability in classical control theory. The notion of differential invariants generalizes barrier certificates [44] : it relaxes the third condition, and also allows more general forms of logical assertions as potential invariants.Verification of safety properties based on identifying inductive invariants avoids the iterative calculation of reachable state sets and is not limited to linear systems. The tool KeYmaera offers support to prove correctness of hybrid systems using deductive verification [43, 44] . To check whether a given polynomial certificate satisfies all the conditions necessary for it to be a barrier certificate, the tool needs to perform symbolic differentiation, and calculations such as simplification and quantifier elimination, with formulas in the theory of reals with arithmetic operators. To fully automate deductive verification, such a tool needs to automatically generate candidates for inductive invariants, and this remains an active area of research (see [28, 50] for automatic generation of invariants by instantiating templates and [45] for generating invariants by fixpoint computation).An abstraction A of a model M is a "simplified" model obtained from M such that proving safety and temporal properties of A is a sufficient condition for proving the corresponding properties of M . Abstraction is an effective strategy for scalability of verification tools, provided there is a way to compute A from M in a tool-supported manner, and a way to refine the current abstraction A if it is not adequate to prove the desired properties. In the case of hybrid systems, the simplicity of the abstract model A can be of various forms: A can be discrete while M is continuous; A can have linear dynamics while M has non-linear dynamics; and A can be a linear model of dimensionality lower than that of M . There is an extensive literature on automatic abstraction of hybrid systems. We note three representative examples.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O