Appeared in AAAI Workshop on Knowledge Discovery in Databases , Eds . Usama M. Fayyad and Ramasamy Uthurusamy , pages 181 -LCB- 192 , Seattle , Washington , July 1994 . Efficient Algorithms for Discovering Association Rules Heikki Mannila Hannu Toivonen ? A. Inkeri Verkamo University of Helsinki , Department of Computer Science P.O. Box 26 -LRB- Teollisuuskatu 23 -RRB- , FIN-00014 Helsinki , Finland e-mail : fmannila , htoivone , verkamog@cs.helsinki.fi Abstract Association rules are statements of the form \ for 90 % of the rows of the relation , if the row has value 1 in the columns in set W , then it has 1 also in column B '' . Agrawal , Imielinski , and Swami introduced the problem of mining association rules from large collections of data , and gave a method based on successive passes over the database . We give an improved algorithm for the problem . The method is based on careful combinatorial analysis of the information obtained in previous passes ; this makes it possible to eliminate unnecessary candidate rules . Experiments on a university course enrollment database indicate that the method outperforms the previous one by a factor of 5 . We also show that sampling is in general a very efficient way of finding such rules . 1 Introduction Data mining -LRB- database mining , knowledge discovery in databases -RRB- has recently been recognized as a promising new field in the intersection of databases , artificial intelligence , and machine learning -LRB- see , e.g. , -LSB- 11 -RSB- -RRB- . The area can be loosely defined as finding interesting rules or exceptions from large collections of data . Recently , Agrawal , Imielinski , and Swami introduced a class of regularities , association rules , and gave an algorithm for finding such rules from a database with binary data -LSB- 1 -RSB- . An association rule is an expression W -RRB- B , where W is a set of attributes and B a single attribute . The intuitive meaning of such a rule is that in the rows of the database where the attributes in W have value true , also the attribute B tends to have value true . For instance , a rule might state that students taking courses CS101 and CS120 , often also take the course CS130 . This sort of information can be used , e.g. , in assigning classrooms for the courses . Applications of association rules include customer behavior ? On leave from Nokia Research Center . 181 analysis for example in a supermarket or banking environment , and telecommunications alarm diagnosis and prediction . In this paper we study the properties of association rule discovery in relations . We give a new algorithm for the problem that outperforms the method in -LSB- 1 -RSB- by a factor of 5 . The algorithm is based on the same basic idea of repeated passes over the database as the method in -LSB- 1 -RSB- . The difference is that our algorithm makes careful use of the combinatorial information obtained from previous passes and in this way avoids considering many unnecessary sets in the process of finding the association rules . Our experimental data consists of two databases , namely university course enrollment data and the fault management database of a switching network . The empirical results show a good , solid performance for our method . A same type of improvement has independently been suggested in -LSB- 2 -RSB- . We also study the theoretical properties of the problem of finding the association rules that hold in a relation . We give a probabilistic analysis of two aspects of the problem , showing that sampling is an efficient way of finding association rules , and that in random relations almost all association rules are small . We also give a simple information-theoretic lower bound for finding one rule , and show that an algorithm suggested by Loveland in -LSB- 7 -RSB- in a different framework actually meets this lower bound . The rest of this paper is organized as follows . Section 2 introduces the problem and the notations . Section 3 describes our algorithm for finding association rules . The analysis of sampling is given in Section 4 . Empirical results and a comparison to the results of -LSB- 1 -RSB- are given in Section 5 . Section 6 is a short conclusion . Appendix A contains the probabilistic analyses of random relations and the lower bound result . Appendix B gives an overview of the implementation . We refer to -LSB- 1 -RSB- for references about related work . 2 Problem First we introduce some basic concepts , using the formalism presented in -LSB- 1 -RSB- . Let R = fI1 ; I2 ; : : : ; Img be a set of attributes , also called items , over the binary domain f0 ; 1g . The input r = ft1 ; : : : ; tng for the data mining method is a relation over the relation schema fI1 ; I2 ; : : : ; Img , i.e. , a set of binary vectors of size m. Each row can be considered as a set of properties or items -LRB- that is , t -LSB- i -RSB- = 1 , Ii 2 t -RRB- . Let W ? R be a set of attributes and t 2 r a row of the relation . If t -LSB- A -RSB- = 1 for all A 2 W , we write t -LSB- W -RSB- = ? 1 . An association rule over r is an expression W -RRB- B , where W ? R and B 2 R n W . Given real numbers -LRB- confidence threshold -RRB- and oe -LRB- support threshold -RRB- , we say that r satisfies W -RRB- B with respect to and oe , if jfi j ti -LSB- W B -RSB- = ? 1gj > = oen and jfi j ti -LSB- W B -RSB- = ? 1gj jfi j ti -LSB- W -RSB- = ? 1gj > = : That is , at least a fraction oe of the rows of r have 1 's in all the attributes of WB , and at least a fraction of the rows having a 1 in all attributes of W also have a 1 in B. Given 182 a set of attributes X , we say that X is covering1 -LRB- with respect to the database and the given support threshold oe -RRB- , if jfi j ti -LSB- X -RSB- = ? 1gj > = oen : That is , at least a fraction oe of the rows in the relation have 1 's in all the attributes of X . As an example , suppose support threshold oe = :3 and confidence threshold = :9 , and consider the example database ABCD ; ABEFG ; ABHIJ ; BK : Now , three of four rows contain the set fABg , so the support is jfi j ti -LSB- AB -RSB- = ? 1gj = 4 = :75 ; supports of A , B , and C are 0.75 , 1 , and 0.25 , correspondingly . Thus , fAg , fBg , and fABg have supports larger than the threshold oe = :3 and are covering , but fCg is not . Further on , the database satisfies fAg -RRB- B , as fABg is covering , and the confidence :75 :75 is larger than = :9 . The database does not satisfy fBg -RRB- A because the confidence :751 is less than the threshold . The coverage is monotone with respect to contraction of the set : if X is covering and B ? X , then ti -LSB- B -RSB- = ? 1 for any i 2 fi j ti -LSB- X -RSB- = ? 1g , and therefore B is also covering . On the other hand , association rules do not have monotonicity properties with respect to expansion or contraction of the left-hand side : if W -RRB- B holds , then WA -RRB- B does not necessarily hold , and if WA -RRB- B holds , then W -RRB- B does not necessarily hold . In the first case the rule WA -RRB- B does not necessarily have sufficient support , and in the second case the rule W -RRB- B does not necessarily hold with sufficient confidence . 3 Finding association rules 3.1 Basic algorithm The approach in -LSB- 1 -RSB- to finding association rules is to first find all covering attribute sets X , and then separately test whether the rule X n fBg -RRB- B holds with sufficient confidence .2 We follow this approach and concentrate on the algorithms that search for covering subsets . To know if a subset X ? R is not covering , one has to read at least a fraction 1 ? oe of the rows of the relation , that is , for small values of support threshold oe almost all of the relation has to be considered . During one pass through the database we can , of course , check for several subsets whether they are covering or not . If the database is large , it is important to make as few passes over the data as possible . The extreme method would be to do just one pass and check for each of the 2m subsets of R whether they are covering or not . This is infeasible for all but the smallest values of m. 1Agrawal et al. use the term large -LSB- 1 -RSB- . 2It is easy to see that this approach is in a sense optimal : the problem of finding all covering subsets of R can be reduced to the problem of finding all association rules that hold with a given confidence . Namely , if we are given a relation r , we can find the covering sets by adding an extra column B with all 1 's to r and then finding the association rules that have B on the right-hand side and hold with certainty 1 . 183 The method of -LSB- 1 -RSB- makes multiple passes over the database . During a database pass , new candidates for covering sets are generated , and support information is collected to evaluate which of the candidates actually are covering . The candidates are derived from the database tuples by extending previously found covering sets in the frontier . For each database pass , the frontier consists of those covering sets that have not yet been extended . Each set in the frontier is extended up to the point where the extension is no longer expected to be covering . If such a candidate unexpectedly turns out to be covering , it is included in the frontier for the next database pass . The expected support required for this decision is derived from the frequency information of the items of the set . Originally the frontier contains only the empty set . An essential property of the method of -LSB- 1 -RSB- is that both candidate generation and evaluation are performed during the database pass . The method of -LSB- 1 -RSB- further uses two techniques to prune the candidate space during the database pass . These are briefly described in Appendix B . We take a slightly different approach . Our method tries to use all available information from previous passes to prune candidate sets between the database passes ; the passes are kept as simple as possible . The method is as follows . We produce a set Ls as the collection of all covering sets of size s . The collection Cs +1 will contain the candidates for Ls +1 : those sets of size s + 1 that can possibly be in Ls +1 , given the covering sets of Ls . 1 . C1 : = ffAg j A 2 Rg ; 2 . s : = 1 ; 3 . while Cs < > ; do 4 . database pass : let Ls be the elements of Cs that are covering ; 5 . candidate generation : compute Cs +1 from Ls ; 6 . s : = s + 1 ; 7 . od ; The implementation of the database pass on line 4 is simple : one just uses a counter for each element of Cs . In candidate generation we have to compute a collection Cs +1 that is certain to include all possible elements of Ls +1 , but which does not contain any unnecessary elements . The crucial observation is the following . Recall that Ls denotes the collection of all covering subsets X of R with jXj = s . If Y 2 Ls + e and e > = 0 , then Y includes ? s + e s ? sets from Ls . This claim follows immediately from the fact that all subsets of a covering set are covering . The same observation has been made independently in -LSB- 2 -RSB- . Despite its triviality , this observation is powerful . For example , if we know that L2 = fAB ; BC ; AC ; AE ; BE ; AF ; CGg ; we can conclude that ABC and ABE are the only possible members of L3 , since they are the only sets of size 3 whose all subsets of size 2 are included in L2 . This further means that L4 must be empty . In particular , if X 2 Ls +1 , then X must contain s +1 sets from Ls . Thus a specification for the computation of Cs +1 is to take all sets with this property : Cs +1 = fX ? R j jXj = s + 1 and X includes s + 1 members of Lsg -LRB- 1 -RRB- 184 This is , in fact , the smallest possible candidate collection Cs +1 in general . For any Ls , there are relations such that the collection of covering sets of size s is Ls , and the collection of coverings sets of size s + 1 is Cs +1 , as specified above .3 The computation of the collection Cs +1 so that -LRB- 1 -RRB- holds is an interesting combinatorial problem . A trivial solution would be to inspect all subsets of size s + 1 , but this is obviously wasteful . One possibility is the following . First compute a collection C s +1 by forming pairwise unions of such covering sets of size s that have all but one attribute in common : C s +1 = fX -LSB- X j X ; X 2 Ls ; jX \ X j = s ? 1g : Then Cs +1 ? C s +1 , and Cs +1 can be computed by checking for each set in C s +1 whether the defining condition of Cs +1 holds . The time complexity is O -LRB- sjLsj2 + jC s +1 jsjLsj -RRB- ; further , jC s +1 j = O -LRB- jLsj2 -RRB- , but this bound is very rough . An alternative method is to form unions of sets from Ls and L1 : C 00s +1 = fX -LSB- X j X 2 Ls ; X 2 L1 ; X 6 ? Xg ; and then compute Cs +1 by checking the inclusion condition . -LRB- Note that the work done in generating Cs +1 does not depend on the size of the database , but only on the size of the collection Ls . -RRB- Instead of computing Cs +1 from Ls , one can compute several families Cs +1 ; : : : ; Cs + e for some e > 1 directly from Ls . The computational complexity of the algorithms can be analyzed in terms of the quantities jLsj , jCsj , jC sj , and the size n of the database . The running time is linear in n and exponential in the size of the largest covering set . For reasons of space we omit here the more detailed analysis . The database passes dominate the running time of the methods , and for very large values of n the algorithms can be quite slow . However , in the next section we shall see that by analyzing only small samples of the database we obtain a good approximation of the covering sets . 4 Analysis of sampling We now consider the use of sampling in finding covering sets . We show that small samples are usually quite good for finding covering sets . Let o be the support of a given set X of attributes . Consider a random sample with replacement of size h from the relation . Then the number of rows in the sample that contain X is a random variable x distributed according to B -LRB- h ; o -RRB- , i.e. , binomial distribution of h trials , each having success probability o . The Chernoff bounds -LSB- 3 , 6 -RSB- state that for all a we have P r -LSB- x > ho + a -RSB- < e ? 2a2 = h : 3Results on the possible relative sizes of Ls and Cs +1 can be found in -LSB- 4 -RSB- . 185 That is , the probability that the estimated support is off by at least is P r -LSB- x > h -LRB- o + -RRB- -RSB- < e ? 2 2h2 = h = e ? 2 2h ; i.e. , bounded by a quantity exponential in h. For example , if = :02 , then for h = 3000 the probability is e ? 2:4 ss :09 . -LRB- Similar results could be even more easily obtained by using the standard normal approximation . -RRB- This means that sampling is a powerful way of finding association rules . Even for fairly low values of support threshold oe , a sample consisting of 3000 rows gives an extremely good approximation of the coverage of an attribute set . Therefore algorithms working in main memory are quite useful for the problem of finding association rules . Appendix A contains an analysis of covering sets in random relations and a lower bound result for the problem of finding association rules . 5 Experiments To evaluate the efficiency of our methods , we compare the original algorithm in -LSB- 1 -RSB- to our algorithm . Candidate generation is performed by extending sets in Ls with other sets in Ls to achieve -LRB- at most -RRB- e-extensions . We compare a less aggressive extending strategy with e = 1 and a more aggressive strategy with e = s , where the size of the candidate sets is doubled during each iteration step . -LRB- We refer to our algorithm as offline candidate determination ; the variants are noted in the following as OCD1 and OCDs . -RRB- In addition to the original algorithm of -LSB- 1 -RSB- -LRB- noted in the following by AISorig -RRB- , we also implemented a minor modification of it that refrains from extending any set with an item that is not a covering set by itself -LRB- noted in the following by AISmod -RRB- . Details about the implementations can be found in Appendix B. 5.1 Data We have used two datasets to evaluate the algorithms . The first is a course enrollment database , including registration information of 4734 students -LRB- one tuple per student -RRB- . Each row contains the courses that a student has registered for , with a total of 127 possible courses . On the average , each row contains 4 courses . A simplified version of the database includes only the students with at least 2 courses -LRB- to be interesting for generating rules -RRB- ; this database consists of 2836 tuples , with an average of 6.5 items per tuple . The figures and tables in this paper represent this latter course enrollment data . The second database is a telephone company fault management database , containing some 30,000 records of switching network notifications . The total number of attributes is 210 . The database is basically a string of events , and we map it to relational form by considering it in terms of overlapping windows . The experiments on this data support the conclusions achieved with the course database . The database sizes we have used are representative for sampling which would result in very good approximations , as was concluded in Section 4 . 186 Size Support oe 0.40 0.20 0.18 0.16 0.14 0.12 0.10 0.08 1 2 11 13 14 14 14 16 18 2 10 17 26 35 53 68 79 3 4 5 12 22 52 102 192 4 1 1 1 5 19 69 171 5 1 19 76 6 1 29 7 3 P 2 26 36 53 76 139 275 568 Table 1 : Number of covering sets . Support oe 0.40 0.20 0.18 0.16 0.14 0.12 0.10 0.08 Count 26 30 48 81 196 544 1426 Max size 4 4 4 4 5 6 7 Table 2 : Number and maximal size of rules -LRB- = :7 -RRB- . 5.2 Results Each algorithm finds , of course , the same covering sets and the same rules . The number of covering sets found with different support thresholds is presented in Table 1 . Correspondingly , the number of association rules is presented in Table 2 ; we used a confidence threshold of 0.7 during all the experiments . The tables show that the number of covering sets -LRB- and rules -RRB- increases very fast with a decreasing support threshold . Figure 1a presents the total time -LRB- in seconds -RRB- as a function of the inverse of the support threshold oe ; we prefer to use the inverse of oe since it corresponds to the common sense idea of the \ rarity '' of the items . Figure 1a shows clearly that OCD is much more time efficient than AIS . The time requirement of OCD is typically 10 -LCB- 20 % of the time requirement of AIS , and the advantage of OCD increases as we lower oe . However , the difference between the algorithms is notable even with a large oe . The difference between the two variants of OCD is not large , and the modification we implemented in AIS did not affect its performance significantly . When we look at the total time as a function of jLj , the number of covering sets -LRB- presented in Figure 1b -RRB- , we observe that both algorithms behave more or less linearly with respect to jLj , but the time requirements of OCD increase much more slowly . As an abstract measure of the amount of database processing we examine the effective volume of the candidates , denoted by Veff . It is the total volume of candidates that are evaluated , weighted by the number of database tuples that must be examined to evaluate them . This measure is representative of the amount of processing needed during the database passes , independent of implementation details . 187 1000 2000 3000 4000 5000 6000 2 4 6 8 10 12 14 Time Inverse of support oe -LRB- a -RRB- AIS OCD1 OCDs q q q q q q q q q q q q q q q q a a a a a a a a a a a a a a a a 1000 2000 3000 4000 5000 6000 100 200 300 400 500 600 Time Number of covering sets -LRB- b -RRB- AIS OCD q q q q q q q q q q q q q q q q a a a a a a a a a a a a a a a a Figure 1 : Total time in seconds -LRB- a -RRB- as a function of the inverse of support and -LRB- b -RRB- as a function of the number of covering sets . 1000 2000 3000 4000 5000 6000 2 4 6 8 10 12 14 Veff Inverse of support oe -LRB- a -RRB- AIS OCD1 OCDs q q q q q q q q q q q q q q q q a a a a a a a a a a a a a a a a 1000 2000 3000 4000 5000 6000 100 200 300 400 500 600 Veff Number of covering sets -LRB- b -RRB- AIS OCD1 OCDs q q q q q q q q q q q q q q q q a a a a a a a a a a a a a a a a Figure 2 : Effective volume of candidates -LRB- a -RRB- as a function of the inverse of support and -LRB- b -RRB- as a function of the number of covering sets . The principal reason for the performance difference of AIS and OCD can be seen in Figures 2a and b . They present the behavior of Veff as the support threshold oe decreases and jLj , the number of covering sets , increases . The number of candidate sets -LRB- and their volume -RRB- considered by OCD is always smaller than that of AIS and the difference increases as more sets are found . These figures also explain why the more aggressively extending variant OCDs does not perform better than the basic OCD1 : even though a more aggressive strategy can reduce the number of passes , it also results in so many more candidates that the reduction in time is offset . Table 3 presents the number of candidates considered by each method . The numbers for AIS are much higher , as it may generate the same candidates over and over again during the database pass . On the other hand , OCD only generates any candidate once -LRB- during the generation time -RRB- and checks that its subsets are covering before evaluating it against the database . While the number of potential candidates generated by OCD is much smaller than that for AIS , still fewer candidates need to be evaluated during the database pass . If sampling is not used or the samples are large , the data does not remain in the 188 Support oe 0.40 0.20 0.18 0.16 0.14 0.12 0.10 0.08 OCD1 128 196 242 289 362 552 950 1756 OCDs 128 217 300 434 625 1084 1799 4137 AISorig 8517 37320 42175 48304 52415 65199 95537 118369 AISmod 9106 38068 42983 48708 53359 66704 96992 120749 Table 3 : Generated candidates . 200 400 600 800 1000 1 2 3 4 5 6 Veff Iteration pass AIS OCD1 q q q q q q q q q q q q q q Figure 3 : Effective volume of candidates during each iteration pass . main memory between passes , but it has to be reloaded for each pass . Thus it would be important to minimize the number of data passes . Also , if we want to overlap the database reads and the processing , the amount of processing performed during each database pass should be small . Figure 3 presents a typical profile of Veff during the passes of one run -LRB- with oe = :1 -RRB- . While the area beneath the curve corresponds to the total volume , the height of the curve at each point describes how much processing is needed during that pass . High peaks correspond to passes where the overlapping of I/O and processing may be endangered if the database is large . Since the confidence threshold affects only the number of rules generated from the covering sets , we have not varied it in our experiments . On the other hand , suitable values for the support threshold oe depend very much on the database . 6 Concluding remarks Association rules are a simple and natural class of database regularities , useful in various analysis or prediction tasks . We have considered the problem of finding the association rules that hold in a given relation . Following the work of -LSB- 1 -RSB- , we have given an algorithm that uses all existing information between database passes to avoid checking the coverage of redundant sets . The algorithm gives clear empirical improvement when compared against the previous results , and it is simple to implement . See also -LSB- 2 -RSB- for similar results . The algorithm can be extended to handle nonbinary attributes by introducing new 189 indicator variables and using their special properties in the candidate generation process . We have also analyzed the theoretical properties of the problem of finding association rules . We showed that sampling is an efficient technique for finding rules of this type , and that algorithms working in main memory can give extremely good approximations . In Appendix A we give some additional theoretical results . We also give a simple lower bound for a special case of the problem , and note that an algorithm from the different framework of -LSB- 7 -RSB- actually matches this bound . We have considered finding association rules from sequential data in -LSB- 8 -RSB- . Several problems remain open . Some of the pruning ideas in -LSB- 1 -RSB- are probably quite useful in certain situations ; recognizing when to use such methods would help in practice . An algorithmic problem is how to find out as efficiently as possible what candidate sets occur in a given database row . Currently we simply check each candidate separately , i.e. , if AB and AC are two candidates , the A entry of each row is checked twice . On certain stages of the search the candidates are heavily overlapping , and it could be useful to utilize this information . A general problem in data mining is how to choose the interesting rules among the large collection of all rules . The use of support and confidence thresholds is one way of pruning uninteresting rules , but some other methods are still needed . In the course enrollment database many of the discovered rules correspond to normal process in the studies . This could be eliminated by considering a partial ordering among courses and by saying that a rule W -RRB- B is not interesting if B is greater than all elements of W with respect to this ordering . References -LSB- 1 -RSB- R. Agrawal , T. Imielinski , and A. Swami . Mining association rules between sets of items in large databases . In Proceedings of the 1993 International Conference on Management of Data -LRB- SIGMOD 93 -RRB- , pages 207 -LCB- 216 , May 1993 . -LSB- 2 -RSB- R. Agrawal and R. Srikant . Fast algorithms for mining association rules in large databases . In VLDB ' 94 , Sept. 1994 . -LSB- 3 -RSB- N. Alon and J. H. Spencer . The Probabilistic Method . John Wiley Inc. , New York , 1992 . -LSB- 4 -RSB- B. Bollob?as . Combinatorics . Cambridge University Press , Cambridge , 1986 . -LSB- 5 -RSB- M. Ellis and B. Stroustrup . The Annotated C++ Reference Manual . Addison-Wesley , 1990 . -LSB- 6 -RSB- T. Hagerup and C. R?ub . A guided tour of Chernoff bounds . Information Processing Letters , 33:305 -LCB- 308 , 1989/90 . -LSB- 7 -RSB- D. W. Loveland . Finding critical sets . Journal of Algorithms , 8:362 -LCB- 371 , 1987 . -LSB- 8 -RSB- H. Mannila , H. Toivonen , and A. I. Verkamo . Association rules in sequential data . Manuscript , July 1994 . 190 -LSB- 9 -RSB- K. Mehlhorn . Data Structures and Algorithms , Volumes 1 -LCB- 3 . Springer-Verlag , Berlin , 1984 . -LSB- 10 -RSB- S. N?aher . LEDA user manual , version 3.0 . Technical report , Max-Planck-Institut f?ur Informatik , Im Stadtwald , D -LCB- 6600 Saarbr?ucken , 1992 . -LSB- 11 -RSB- G. Piatetsky-Shapiro and W. J. Frawley , editors . Knowledge Discovery in Databases . AAAI Press / The MIT Press , Menlo Park , CA , 1991 . A Probabilistic analysis and a lower bound In this appendix , we present some results describing the theoretical properties of the problem of finding association rules . We first first show that in one model of random relations all covering sets are small . Consider a random relation r = ft1 ; : : : ; tng over attributes R = fI1 ; I2 ; : : : ; Img ; assume that each entry ti -LSB- Aj -RSB- of the relation is 1 with probability q and with probability 1 ? q , and assume that the entries are independent . Then the probability that ti -LSB- X -RSB- = ? 1 is qh , where h = jXj . The number x of such rows is distributed according to B -LRB- n ; qh -RRB- , and we can again use the Chernoff bounds to obtain P r -LSB- x > oen -RSB- = P r -LSB- x > nqh + n -LRB- s ? qh -RRB- -RSB- < e ? 2n -LRB- s?qh -RRB- 2 : This is furthermore bounded by e?oen , if oe > = 2qh . -LRB- For oe = :01 and q = :5 , this means h > = 8 , and for oe = :01 and q = :1 it means h > = 3 . -RRB- Now the expected number of covering sets of size h is bounded by mhe?oen . This is less than 0.5 , provided oen > h ln m + ln 2 . Thus a random relation typically has only very few covering sets . Of course , relations occurring in practice are not random . Next we describe some lower bound observations . Note first that a relation with one row consisting of all 1 's satisfies all association rules . Thus the output of an algorithm that produces all association rules holding in a relation can be of exponential size in the number of attributes in the relation . We now give an information-theoretic lower bound for finding one association rule in a restricted model of computation . We consider a model of computation where the only way of getting information from relation r is by asking questions of the form \ is the set X covering '' . This model is realistic in the case the relation r is large and stored using a database system , and the model is also quite close to the one underlying the design of the algorithm in -LSB- 1 -RSB- . Assume the relation r has n attributes . In the worst case one needs at least log ? n k ? ss k log -LRB- n = k -RRB- questions of the form \ is the set X covering '' to locate one maximal covering set , where k is the size of the covering set . The proof of this claim is simple . Consider relations with exactly 1 maximal covering set of size k . There are ? n k ? different possible answers to the problem of finding the 191 maximal covering set . Each question of the form \ is the set X covering '' provides at most 1 bit of information . Loveland -LSB- 7 -RSB- has considered the problem of finding \ critical sets '' . Given a function f : P -LRB- R -RRB- ! f0 ; 1g that is downwards monotone -LRB- i.e. , if f -LRB- X -RRB- = 1 and Y ? X , then f -LRB- Y -RRB- = 1 -RRB- , a set X is critical if f -LRB- X -RRB- = 1 , but f -LRB- Z -RRB- = for all supersets Z of X. Thus maximal covering sets are critical sets of the function f -LRB- X -RRB- = 1 , if X is covering , and f -LRB- X -RRB- = 0 , otherwise . The lower bound above matches exactly the upper bound provided by one of Loveland 's algorithms . The lower bound above can easily be extended to the case where the task is to find k maximal covering sets . An interesting open problem is whether Loveland 's algorithm can be extended to meet the generalized lower bound . B Pruning methods and implementations The method of -LSB- 1 -RSB- uses two techniques to prune the candidate space during the database pass . In the \ remaining tuples optimization '' , the occurrences of each frontier set are counted . A candidate is pruned by the optimization method , when there are less occurrences of the frontier set left than are needed for the candidate to be covering . Remember that the total number of occurrences of a frontier set has been evaluated in earlier passes . In the \ pruning function optimization '' , items are assigned weights based on their rarity , and tuples are assigned weights from their items with synthesized functions . This method prunes a candidate , if | based on its weight | it is so rare that it can not be covering . To know the weight threshold for each set in the frontier , for each candidate set that is not expected to be covering | and thus could be in the frontier in the next database pass | oen highest total weights of database rows containing the set are maintained . The lowest of these values is then stored , and the weights of candidates are compared against this weight threshold of the corresponding frontier set . The success of this method depends on the distributional properties of items . The implementations of the algorithms have been kept reasonably straightforward . Attention was paid to the realization of the ideas rather than to optimizations of time or space . We wrote the algorithms in C++ -LSB- 5 -RSB- , and used data structures from LEDA library -LSB- 10 -RSB- . Implementations of both algorithms use the same basic data structures and algorithms for representing and manipulating sets of attributes . This ensures that timing comparisons are fair . Attributes are represented by their names as strings . Attribute sets are implemented as sorted sequences of strings , and collections of attribute sets , i.e. Ci and Lj , as sorted sequences of attribute sets . The sorted sequences are implemented as -LRB- 2,4 -RRB- - trees -LSB- 9 -RSB- . The above mentioned pruning methods of -LSB- 1 -RSB- require additional data structures . \ Remaining tuples optimization '' only needs a counter for each frontier set , plus checking of the pruning condition when the support count for a candidate is being increased . For \ pruning function optimization '' , the weights of items are stored in a randomized search tree during the first database pass . The weight thresholds are stored in another randomized search tree each database pass , to be utilized in the next iteration . The candidates are pruned by this method as soon as they are created . We have not implemented any memory saving techniques referred to in -LSB- 1 -RSB- that might decrease the precision of the pruning . 192

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O B O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

algorithms
association rules
covering sets
sampling


Improved parallel polynomial division Dario Bini ? Victor Pan ?? Department of Mathematics Dept. of Comp . Sci . University of Pisa Columbia University ; Department of Math . & Comp . Sci . Lehman College , CUNY CUS-029-90 Summary . We compute the first N coefficients of the reciprocal r -LRB- x -RRB- , r -LRB- x -RRB- p -LRB- x -RRB- = 1 mod xN -LRB- given a natural N and a polynomial p -LRB- x -RRB- -RRB- , -LRB- p -LRB- 0 -RRB- < > 0 -RRB- , by using O -LRB- h log N -RRB- arithmetic steps and O ? -LRB- N = h -RRB- -LRB- 1 + 2 ? h log -LRB- h -RRB- N -RRB- ? processors , for any h , h = 1 ; 2 ; ... ; log ? N , under the PRAM arithmetic models , provided that O -LRB- log N -RRB- steps and N processors suffice to perform DFT at N points and that log -LRB- 0 -RRB- N = N , log -LRB- h -RRB- N = log2 log -LRB- h ? 1 -RRB- N , h = 1 ; ... ; log ? N , log ? N = minfh : log -LRB- h -RRB- N < = 1g . The same estimates apply to several other computations such as the division with a remainder of two polynomials of degrees O -LRB- N -RRB- , the inversion of an N ? N triangular Toeplitz matrix , and the evaluation modulo xN of the m-root p -LRB- x -RRB- 1 = m of p -LRB- x -RRB- for any fixed natural m . This improves the known estimates of Reif-Tate and Georgiev . ? Supported by NSF Grant CCR 8805782 and by MPI -LRB- 40 % funds -RRB- . ?? Supported by NSF grant CCR 8805782 and by PSC CUNY Awards # 661340 , # 668541 and # 669210 . 1 . Introduction . In this paper we will improve the known upper estimates for the parallel arithmetic complexity of computing the reciprocal of a polynomial and , consequently , of the equivalent computations , such as polynomial and power series division and triangular Toeplitz matrix inversion -LRB- compare -LSB- BP -RSB- -RRB- . The techniques and the results can be extended to other polynomial computations based on recursive processes , such as Newton 's iteration , in particular , to computing -LRB- modulo xN -RRB- the -LRB- 1 m -RRB- - th power -LRB- or the m-th root -RRB- p -LRB- x -RRB- 1 = m of a polynomial p -LRB- x -RRB- . We will state our estimates in the form OA -LRB- t ; P -RRB- , which means that our computations can be performed by using O -LRB- st -RRB- arithmetic steps and O -LRB- P = s -RRB- processors -LRB- for any fixed s , 1 < = s < = P -RRB- , under the PRAM arithmetic models of computing -LRB- see -LSB- KR -RSB- -RRB- . We will assume the bounds OA -LRB- log m ; m -RRB- on the cost of performing -LRB- forward and inverse -RRB- discrete Fourier transforms at the m-th roots of 1 -LSB- hereafter referred to as DFT -LRB- m -RRB- -RSB- . Hereafter , log stands for log2 -LRB- all logarithms are binary -RRB- . Given a natural N and the coefficients of a polynomial p -LRB- x -RRB- , p -LRB- 0 -RRB- < > 0 , we will compute the first N coefficients of the reciprocal power series , r -LRB- x -RRB- = 1 X i = 0 rixi ; r -LRB- x -RRB- p -LRB- x -RRB- = 1 mod xN ; r0 = 1 p -LRB- 0 -RRB- ; -LRB- 1:1 -RRB- yielding the complexity estimates cN = OA -LRB- h log N ; N h -LRB- 1 + 2 ? h log -LRB- h -RRB- N -RRB- -RRB- ; for all h ; h = 1 ; 2 ; ... ; log ? N : -LRB- 1:2 -RRB- Here , log -LRB- 0 -RRB- N = N ; log -LRB- h -RRB- N = log2 log -LRB- h ? 1 -RRB- N ; h = 1 ; ... ; log ? N ; log ? N = minfh : log -LRB- h -RRB- N < = 1g : In particular , for h = log ? N , -LRB- 1.2 -RRB- turns into cN = OA -LRB- log ? N log N ; N = log ? N -RRB- : -LRB- 1:3 -RRB- This improves the previous record bounds of -LSB- G -RSB- , cN = OA -LRB- log N ; N log2 N -RRB- ; and of -LSB- RT -RSB- , cN = OA -LRB- log N log log N ; N = log log N -RRB- : -LRB- See -LSB- BP -RSB- and -LSB- BPa -RSB- for other previous works . -RRB- 2 . Some known algorithms . It is well known that , given a polynomial p -LRB- x -RRB- , p -LRB- 0 -RRB- < > 0 , the first m coefficients of the reciprocal power series r -LRB- x -RRB- of -LRB- 1.1 -RRB- can be computed by performing dlog me steps of Newton 's iteration , z0 -LRB- x -RRB- = 1 =p -LRB- 0 -RRB- ; zi +1 -LRB- x -RRB- = zi -LRB- x -RRB- -LRB- 2 ? p -LRB- x -RRB- zi -LRB- x -RRB- -RRB- ; i = 1 ; ... ; dlog me : -LRB- 2:1 -RRB- In fact , since r -LRB- x -RRB- ? zi +1 -LRB- x -RRB- = ? r -LRB- x -RRB- ? zi -LRB- x -RRB- ? 2 p -LRB- x -RRB- ; -LRB- 2:2 -RRB- the number of coefficients shared by r -LRB- x -RRB- and zi -LRB- x -RRB- is doubled in each iteration step i , so that , for all i , zi -LRB- x -RRB- = r -LRB- x -RRB- mod x2i : -LRB- 2:3 -RRB- The algorithm of Sieveking-Kung is based on this scheme , where , however , both polynomials p -LRB- x -RRB- and zi -LRB- x -RRB- of -LRB- 2.1 -RRB- are reduced modulo x2i ; then step i of Newton 's process outputs a polynomial of degree at most 3 -LRB- 2i ? 1 -RRB- , whose coefficients can be computed by means of forward and inverse DFTs at m -LRB- i -RRB- points , m -LRB- i -RRB- > 3 -LRB- 2i ? 1 -RRB- , for the cost OA -LRB- i ; 2i -RRB- . This implies the bound OA -LRB- log2 n ; n = log n -RRB- on the overall cost of computing the coefficients of r -LRB- x -RRB- mod xn , whicn turns into the bound cn ; k = OA -LRB- log n log -LRB- n k -RRB- ; n = log -LRB- n k -RRB- -RRB- -LRB- 2:4 -RRB- if we initially know the k coefficients of r -LRB- x -RRB- mod xk . In -LSB- G -RSB- it is pointed out that the relation -LRB- 2.3 -RRB- still holds even if p -LRB- x -RRB- in -LRB- 2.1 -RRB- is replaced by p -LRB- x -RRB- mod x2i . Then the degrees of the polynomials zi -LRB- x -RRB- are bounded by i2i , i = ; 1 ; ... ; and thus , the coefficients of zd -LRB- x -RRB- , for d = dlog ne , can be computed by means of the single inverse DFT at O -LRB- n log n -RRB- points -LRB- roots of 1 -RRB- once the values of zd -LRB- x -RRB- at these points have been computed . To compute the values of zd -LRB- x -RRB- at a set of points , we may compute the values of pi -LRB- x -RRB- = p -LRB- x -RRB- mod x2i at this set and then , for each point , recursively apply the equation -LRB- 2.1 -RRB- for i = ; 1 ; ... ; d ? 1 , thus avoiding the application of DFT . This way , the n coefficients of r -LRB- x -RRB- and xn can be computed for the cost OA -LRB- log n ; n log2 n -RRB- -LRB- -LSB- G -RSB- -RRB- . 3 . Improved algorithms . In this section , we will combine the two ways of applying Newton 's iteration -LRB- 2.1 -RRB- , that is , the coefficientwise way -LRB- Sieveking-Kung -RRB- and the pointwise way -LRB- -LSB- G -RSB- -RRB- , in order to decrease the overall parallel computational cost . In particular , we will consider Newton 's iteration that starts with y0 -LRB- x -RRB- = r -LRB- x -RRB- mod xk -LRB- 3:1 -RRB- for any fixed natural k -LRB- not necessarily for k = 1 -RRB- and that continues as follows : pi -LRB- x -RRB- = p -LRB- x -RRB- mod xk2i +1 ; -LRB- 3:2 -RRB- yi +1 -LRB- x -RRB- = yi -LRB- x -RRB- ? 2 ? pi -LRB- x -RRB- yi -LRB- x -RRB- ? ; -LRB- 3:3 -RRB- i = ; 1 ; ... . Then -LRB- 2.2 -RRB- immediately implies that yi -LRB- x -RRB- = r -LRB- x -RRB- mod xk2i ; -LRB- 3:4 -RRB- and we verify by induction on i that deg yi -LRB- x -RRB- < = -LRB- i + 1 -RRB- k 2i ? 2i +1 + 1 : -LRB- 3:5 -RRB- -LRB- 3.4 -RRB- and -LRB- 3.5 -RRB- will immediately imply the correctness of our basis algorithm below . Now , let s denote an integer , 4 < = s < = n ; let log log s and log n be integers . Basis algorithm . Input : integers n and s and the coefficients of p -LRB- x -RRB- and r -LRB- x -RRB- mod xn = s. Output : the coefficients of r -LRB- x -RRB- mod xn = log s. Initialize : set k = n = s , j = log s ? 2 log log s , q = n = log s , y0 -LRB- x -RRB- = r -LRB- x -RRB- mod xk . Computation : 1 . j times apply DFT at q points , to evaluate the polynomials y0 -LRB- x -RRB- of -LRB- 3.1 -RRB- and pi -LRB- x -RRB- of -LRB- 3.2 -RRB- , for i = ; ... ; j ? 1 , at the q-th roots of 1 . 2 . Use the equations -LRB- 3.3 -RRB- , for i = ; ... ; j ? 1 , in order to evaluate yi -LRB- x -RRB- , for i = 1 ; ... ; j , at the q-th roots of 1 . 3 . Apply the inverse DFT at q points in order to evaluate the coefficients of yj -LRB- x -RRB- . -LSB- yj -LRB- x -RRB- = r -LRB- x -RRB- mod xk2j , due to -LRB- 3.4 -RRB- , and k2j = n = log2 s. -RSB- 4 . Update the initialized parameters by first setting y0 -LRB- x -RRB- = yj -LRB- x -RRB- mod xk2j , q = n log log s = log s , and then k = n = log2 s , j = log log s ; then redefine pi ? 1 -LRB- x -RRB- and yi -LRB- x -RRB- , for i = 1 ; ... ; j , by using the equations -LRB- 3.2 -RRB- and -LRB- 3.3 -RRB- ; then again apply steps 1-3 to compute and to output the coefficients of the polynomial yj -LRB- x -RRB- -LRB- which equals r -LRB- x -RRB- mod xk2j , where k2j = n = log s for the updated k and j -RRB- . The cost of the computation by the basis algorithm is clearly bounded by OA -LRB- log n ; n -RRB- , and we will recursively apply this algorithm for a fixed n and for s successively taking the values s0 = n ; s1 = log n ; s2 = log log n ; ... sh ? 1 = log -LRB- h ? 1 -RRB- n ; -LRB- 3:6 -RRB- for 1 < = h < = log ? n -LRB- for simplicity , let log -LRB- g -RRB- n be integers for g = 1 ; ... ; h -RRB- . This way , we will compute the first m -LRB- h ; n -RRB- = n = log -LRB- h -RRB- n coefficients of r -LRB- x -RRB- for the cost OA -LRB- h log n ; n -RRB- . Note that m -LRB- h ; n -RRB- = n if h = log ? n ; otherwise m -LRB- h ; n -RRB- < n , and then , to compute the remaining coefficients of r -LRB- x -RRB- mod xn , we set y0 -LRB- x -RRB- = r -LRB- x -RRB- mod xm -LRB- h ; n -RRB- ; k = m -LRB- h ; n -RRB- ; j = dlog -LRB- h +1 -RRB- ne ; -LRB- 3:7 -RRB- set q = -LRB- j + 1 -RRB- k2j ? 2j +1 + 1 -LRB- 3:8 -RRB- and apply steps 1-3 of the computation of the basis algorithm . The complexity of these steps is upper bounded by OA -LRB- log n ; n -LRB- log -LRB- h +1 -RRB- n -RRB- 2 -RRB- , which we may replace by the weaker bound OA -LRB- h log n ; n h log -LRB- h -RRB- n -RRB- . The overall cost of computing the coefficients of r -LRB- x -RRB- mod xn is thus bounded by OA -LRB- h log n ; n -LRB- 1 + log -LRB- h -RRB- n h -RRB- -RRB- , which turns into the bound c?N ; h = OA -LRB- h log N ; N 2 ? h -LRB- 1 + log -LRB- h -RRB- N h -RRB- -RRB- -LRB- 3:9 -RRB- if we denote N = n 2h . Finally , we apply h steps of the Sieveking-Kung algorithm to compute the coefficients of r -LRB- x -RRB- mod xN , that have not yet been computed ; the cost of this step , OA -LRB- h log N ; N = h -RRB- -LSB- see -LRB- 2.4 -RRB- -RSB- , is smaller than the bound -LRB- 3.9 -RRB- , and thus we arrive at -LRB- 1.2 -RRB- , for h < = log ? n , and surely , log ? N = O -LRB- log ? n -RRB- as N ! 1 , so that we extend -LRB- 1.2 -RRB- to all h < = log ? N and then deduce -LRB- 1.3 -RRB- . To relax the assumption about the integrality of log log s and log -LRB- g -RRB- n for g = 1 ; ... ; h , we just need to replace the values of k and j in the basis algorithm -LRB- chosen at the initialization step and at step 4 -RRB- and in -LRB- 3.7 -RRB- , as well as the values of s1 ; s2 ; ... ; sh in -LRB- 3.6 -RRB- and m -LRB- h ; n -RRB- , by the minimum integers exceeding these values and to define q by -LRB- 3.8 -RRB- . It is easy to verify that such a replacement preserves -LRB- 3.9 -RRB- . Let us finally show a simple extension of our computation that also leads to -LRB- 3.9 -RRB- -LSB- and therefore , to -LRB- 1.2 -RRB- and -LRB- 1.3 -RRB- -RSB- . Denote log -LRB- 2 ; 0 -RRB- n = n ; log -LRB- 2 ; a -RRB- n = log ? log -LRB- 2 ; a ? 1 -RRB- n ? 2 ; s = 1 ; 2 ; ... ; log -LRB- 2 ; ? -RRB- n ; log -LRB- 2 ; ? -RRB- n = minfa : log -LRB- 2 ; a -RRB- n < = 1g : Recursively apply steps 1-3 of the basis algorithm , for g = 1 ; ... ; G , G < = log -LRB- 2 ; ? -RRB- n , G = O -LRB- h -RRB- , where in the g-th call to these steps , q , j and k are replaced by q -LRB- g -RRB- , j -LRB- g -RRB- , and k -LRB- g -RRB- , respectively , q -LRB- g -RRB- = -LRB- j -LRB- g -RRB- + 1 -RRB- k -LRB- g -RRB- 2 -LRB- j -LRB- g -RRB- ? 2j -LRB- g -RRB- +1 + 2 , j -LRB- g -RRB- = dlog k -LRB- g -RRB- k -LRB- g ? 1 -RRB- e , for all g ; k -LRB- G -RRB- = n , and k -LRB- g -RRB- = n = log -LRB- 2 ; g -RRB- n , for j = ; 1 ; ... ; G ? 1 . References -LSB- BP -RSB- Bini , D. , and V. Pan , Polynomial division and its computational complexity , Jour . Complexity , 2 -LRB- 1986 -RRB- 179-203 . -LSB- BPa -RSB- Bini , D. , and V. Pan , On polynomial and matrix computation , to appear . -LSB- G -RSB- Georgiev , R. E. , Inversion of triangular Toeplitz matrices by using the fast Fourier transform , J. New Gener . Comput . Sys. 2 -LRB- 3 -RRB- -LRB- 1989 -RRB- 247-256 . -LSB- KR -RSB- Karp , R. M. and V. Ramachandran , A survey of parallel algorithms for shared memory machines , Report No . UCB/CSD 88/408 , Comp . Sci . Div . -LRB- EECS -RRB- , U. California , Berkeley , CA -LRB- 1988 -RRB- , to appear in Handbook of Theor . Comp . Science , North-Holland , 1990 . -LSB- RT -RSB- Reif , J. H. , and S. H. Tate , Optimal size division circuits , Proc . 21-th Ann . ACM Symp . on Theory of Computing -LRB- 1989 -RRB- 264-270 .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

computational complexity
parallel algorithms
polynomial division
triangular toeplitz matrices


A Near-linear-time Approximation Algorithms for Maximum-leaf Spanning Tree Hsueh-I Lu and R. Ravi Department of Computer Science Brown University Providence , Rhode Island 02912 CS-96-06 January 1996 A Near-linear-time Approximation Algorithm for Maximum-leaf Spanning Tree Hsueh-I Lu hil@cs.brown.edu Department of Computer Science Brown University R. Ravi ravi+@andrew.cmu.edu Graduate School of Industrial Administration Carnegie Mellon University January 1996 Abstract Given an undirected graph G , finding a spanning tree of G with maximum number of leaves is not only NP-complete -LSB- 11 -RSB- but also MAX SNP-complete -LSB- 10 -RSB- . The approximation ratio of the previously best known approximation algorithm for maximum leaf spanning tree is three -LSB- 19 -RSB- . However , the high-order running time required by the previous algorithm makes it impractical . In this paper we give a new factor-three approximation algorithm for the same problem . The running time required by our algorithm is almost linear in the size of G . This improves the previous algorithm by a factor of ~ ? -LRB- mn3 -RRB- , where n is the number of nodes and m is the number of edges . 1 Introduction The NP-complete Maximum Leaf Spanning Tree problem has been extensively studied -LSB- 12 , 17 , 23 , 25 , 7 , 3 , 19 -RSB- . It has applications in circuit layouts and communication networks -LSB- 5 , 27 -RSB- . In -LSB- 19 -RSB- a series of approximation algorithms are given . The approximation ratios of the first two algorithms in the series are shown to be five and three . The observation leads to an open question : \ is the series of algorithms a polynomial-time approximation scheme -LRB- PTAS -RRB- -LSB- 14 -RSB- ? '' The open question is answered negatively in -LSB- 10 -RSB- , which shows that Maximum Leaf Spanning Tree is MAX SNP - complete -LSB- 22 -RSB- . Therefore there exists some constant ffl > such that there is no -LRB- 1 + ffl -RRB- - approximation for Maximum Leaf Spanning Tree unless P = NP -LSB- 1 , 2 -RSB- . The series of approximation algorithms given in -LSB- 19 -RSB- is based on the technique of local optimization -LSB- 4 , 16 , 18 , 21 , 28 , 8 , 24 , 9 , 13 , 6 -RSB- . The kth algorithm in the series uses k-changes to achieve local optimality . The time complexity , O -LRB- mknk +1 -RRB- , is intolerably high even if k is small . In this paper we give an efficient approximation algorithm for Maximum Leaf Spanning Tree with the best currently achievable approximation ratio for the problem . 2 Preliminaries Let G be an undirected graph . We use V -LRB- G -RRB- to denote the set of nodes in G . We refer to an edge uw of G as the edge incident to u and w. Let G0 be a subgraph of G . A node v is in G0 if v 2 V -LRB- G0 -RRB- . An edge uw is incident to G0 if exactly one of u and w is in V -LRB- G0 -RRB- . The degree of v in G0 is the number of edges of G0 incident to v. Let Vi -LRB- G0 -RRB- denote the set of nodes that have degree i in G0 . Let ? Vi -LRB- G0 -RRB- denote the set of nodes that have degree at least i in G0 . Clearly ? V0 -LRB- G0 -RRB- = V -LRB- G -RRB- . Let T be a subtree of G . One can easily verify that j ? V3 -LRB- T -RRB- j < = jV1 -LRB- T -RRB- j ? 2 : -LRB- 1 -RRB- The following lemma is also straightforward . Lemma 1 Let T be a subtree of G. Let u , v , and w be three distinct nodes of G . If v is in pathT -LRB- u ; w -RRB- , then v is not a leaf of T . 3 Leafy Subtrees and Leafy Forests Let T be a subtree of G . We say T is leafy if ? V3 -LRB- T -RRB- is not empty , and every node in V2 -LRB- T -RRB- is adjacent to exactly two nodes in ? V3 -LRB- T -RRB- . A forest F of G is leafy if F is composed of disjoint leafy subtrees of G . We say F is maximally leafy if F is not a subgraph of any other leafy forest of G . In this section we show the following theorem . Theorem 1 Let F be a maximally leafy forest of G. Let T be a spanning tree of G such that F is a subgraph of T . Then jV1 -LRB- T -RRB- j > = jV1 -LRB- ^ T -RRB- j = 3 for any spanning tree ^ T of G . The following lemma ensures that at least one-third of the nodes in a leafy subtree T are leaves of T . Lemma 2 Let T be a leafy subtree of G . Then jV -LRB- T -RRB- j < = 3jV1 -LRB- T -RRB- j ? 5 . Proof Since T is leafy , each node in V2 -LRB- T -RRB- must be adjacent in T to exactly two nodes in ? V3 -LRB- T -RRB- . Therefore jV2 -LRB- T -RRB- j < = j ? V3 -LRB- T -RRB- j ? 1 . It follows from -LRB- 1 -RRB- that jV -LRB- T -RRB- j = jV1 -LRB- T -RRB- j + jV2 -LRB- T -RRB- j + j ? V3 -LRB- T -RRB- j < = jV1 -LRB- T -RRB- j + 2j ? V3 -LRB- T -RRB- j ? 1 < = 3jV1 -LRB- T -RRB- j ? 5 : Let F be a maximally leafy forest of G. Let T1 ; : : : ; Tk be the disjoint leafy subtrees of G in F . One can easily verify that F has the following properties . The properties for maximally leafy forests 1 . Let w be a node in ? V2 -LRB- Ti -RRB- . Then w can not be adjacent in G to any node not in Ti . 2 . Let w be a node in Ti . Let w1 and w2 be two distinct nodes adjacent to w in G . If w1 is not in F , then w2 must be in Ti . 3 . Let w be a node not in F . If w is adjacent to two distinct nodes not in F , then the degree of w in G is two . 4 . Let w be a node not in F . Let w1 , w2 , and w3 be three distinct nodes adjacent to w in G . If w1 62 F , then w2 and w3 must be in the same leafy subtree Ti for some i 2 f1 ; : : : ; kg . The following two lemmas are essential to proving the theorem . Lemma 3 Let F be a maximally leafy forest of G that is composed of k disjoint leafy subtrees T1 ; : : : ; Tk of G . Then jV1 -LRB- ^ T -RRB- j < = jV -LRB- F -RRB- j ? k + 1 ; for any spanning tree ^ T of G. Proof Let vi be a node in ? V3 -LRB- Ti -RRB- for every i = 1 ; : : : ; k. For every i = 1 ; : : : ; k ? 1 , let ui be the node in Ti that is farthest from vi in path ^ T -LRB- vi ; vk -RRB- . Let ^ v1 ; : : : ; ^ v ` be the distinct nodes in V1 -LRB- ^ T -RRB- n V -LRB- F -RRB- . Namely each ^ vj is a leaf of ^ T not in F . For every j = 1 ; : : : ; ` , let ^ uj be the node in F that is closest to ^ vj in path ^ T -LRB- ^ vj ; vk -RRB- . We show u1 ; : : : ; uk ? 1 ; ^ u1 ; : : : ; ^ u ` are ` + k ? 1 distinct nodes in V -LRB- F -RRB- n V1 -LRB- ^ T -RRB- , which implies the lemma . By definition of ui we know the first edge from ui in path ^ T -LRB- ui ; vk -RRB- is in ? -LRB- Ti -RRB- . By Property 1 we know ui 2 V1 -LRB- Ti -RRB- . Therefore ui < > vi and ui < > vk . It follows from Lemma 1 that ui 62 V1 -LRB- ^ T -RRB- , since ui 2 path ^ T -LRB- vi ; vk -RRB- . Hence ui 2 V -LRB- F -RRB- n V1 -LRB- ^ T -RRB- . Since T1 ; : : : ; Tk are disjoint and ui 2 Ti for every i = 1 ; : : : ; k ? 1 , we know u1 ; : : : ; uk ? 1 are k ? 1 distinct nodes in V -LRB- F -RRB- n V1 -LRB- ^ T -RRB- . Let j be one of 1 ; : : : ; ` . By definition of ^ vj we know that the first edge from ^ uj in path ^ T -LRB- ^ uj ; ^ vj -RRB- is in ? -LRB- Tj ? -RRB- for some Tj ? that contains ^ uj . By Property 1 we know ^ u 2 V1 -LRB- Tj ? -RRB- . Therefore ^ uj < > ^ vj and ^ uj < > vk . It follows from Lemma 1 that ^ uj 62 V1 -LRB- ^ T -RRB- , since ^ uj 2 path ^ T -LRB- ^ uj ; vk -RRB- . Hence ^ uj 2 V -LRB- F -RRB- n V -LRB- ^ T -RRB- . We show that ^ u1 ; : : : ; ^ u ` are distinct , and ^ uj 62 fu1 ; : : : ; uk ? 1g . Assume for a contradiction that ^ uj = ^ uj0 for some j0 < > j. Let P = path ^ T -LRB- ^ uj ; ^ vj -RRB- and P = path ^ T -LRB- ^ uj0 ; ^ vj0 -RRB- . let w be the node in V -LRB- P -RRB- \ V -LRB- P -RRB- that is closest to ^ vj in P . Since ^ vj 62 P , there exists a node w1 in path ^ T -LRB- w ; ^ vj -RRB- such that ww1 is an edge of P . Since ^ vj0 62 P , there exists a node w2 in path ^ T -LRB- w ; ^ vj0 -RRB- such that ww2 is an edge of P . Clearly w1 < > w2 and w1 ; w2 62 F . It follows from Property 2 that w < > ^ uj , implying that w 62 F . Therefore there exists an edge ww3 in P where w3 < > w1 and w3 < > w2 . This contradicts the fact that F is maximally leafy by Property 3 . Assume for a contradiction that ^ uj = ui for some i 2 f1 ; : : : ; k ? 1g . Let P = path ^ T -LRB- ^ uj ; ^ vj -RRB- and P = path ^ T -LRB- ui ; vk -RRB- . Let w be the node in V -LRB- P -RRB- \ V -LRB- P -RRB- that is closest to ^ vj in P . Since ^ vj 62 P , there exists a node w1 in path ^ T -LRB- w ; ^ vj -RRB- such that ww1 is an edge of P . Since vk 62 P , there exists a node w2 in path ^ T -LRB- w ; vk -RRB- such that ww2 is an edge of P . Clearly w1 < > w2 and w1 62 F . By definition of ui we know w2 62 Ti . It follows from Property 2 that w 62 Ti . Therefore there exists an edge ww3 in P where w3 < > w1 and w3 < > w2 . By Property 3 we know w3 2 F , implying w3 2 Ti . This contradicts the fact that F is maximally leafy by Property 4 , since w1 62 F , w2 62 Ti , and w3 2 Ti . Lemma 4 Let F be a forest of G that has k disjoint nonsingleton subtrees . Let T be a spanning tree of G such that F is a subgraph of T . Then jV1 -LRB- T -RRB- j > = jV1 -LRB- F -RRB- j ? 2 -LRB- k ? 1 -RRB- . Proof Let F be the forest of G obtained from F by adding an edge e in T n F to F . Let k0 be the number of disjoint nonsingleton subtrees of F . We show jV1 -LRB- F -RRB- j ? 2 -LRB- k0 ? 1 -RRB- > = jV1 -LRB- F -RRB- j ? 2 -LRB- k ? 1 -RRB- : -LRB- 2 -RRB- The lemma thus follows inductively , since the number of disjoint nonsingleton subtrees in T is one . ffl If e is not incident to any nonsingleton subtree of F , then k0 = k +1 and jV1 -LRB- F -RRB- j = jV1 -LRB- F -RRB- j +2 . Thus -LRB- 2 -RRB- holds . ffl If e is incident to exactly one nonsingleton subtree of F , then k0 = k and jV1 -LRB- F -RRB- j > = jV1 -LRB- F -RRB- j. Thus -LRB- 2 -RRB- holds . ffl If e is incident to two nonsingleton subtrees of F , then k0 = k ? 1 and jV1 -LRB- F -RRB- j > = jV1 -LRB- F -RRB- j ? 2 . Thus -LRB- 2 -RRB- holds . We are now ready to prove the theorem as follows . Proof for Theorem 1 Suppose F has k disjoint leafy subtrees T1 ; : : : ; Tk . By Lemma 2 we know jV -LRB- F -RRB- j = k X i = 1 jV -LRB- Ti -RRB- j < = 3jV1 -LRB- F -RRB- j ? 5k : It follows from Lemma 3 , the above inequality , and Lemma 4 that jV1 -LRB- ^ T -RRB- j < = jV -LRB- F -RRB- j ? k + 1 < = 3jV1 -LRB- F -RRB- j ? 6k + 1 < = 3 -LRB- jV1 -LRB- T -RRB- j + 2 -LRB- k ? 1 -RRB- -RRB- ? 6k + 1 < = 3jV1 -LRB- T -RRB- j : 4 The Algorithm We give an approximation algorithm for Maximum Leaf Spanning Tree in this section . Given a graph G , our algorithm computes a spanning tree T for G by the following two steps . 1 . Obtain a maximally leafy forest F for G. 2 . Add edges to F to make it a spanning tree T for G . It follows from Theorem 1 that the approximation ratio of the above algorithm is three , which the same as the algorithm in -LSB- 19 -RSB- . We show that our algorithm can be implemented to run in time O -LRB- -LRB- m + n -RRB- ff -LRB- m ; n -RRB- -RRB- . We implement the first step of our algorithm as follows . For every node v in G do Create a set S -LRB- v -RRB- that contains only v. Let d -LRB- v -RRB- be zero . For every node v in G do Let S0 be an empty set . Let d0 be zero . For every node u that is adjacent to v in G do If u 62 S -LRB- v -RRB- and S -LRB- u -RRB- 62 S0 then Increase d0 by one . Insert S -LRB- u -RRB- into S0 . If d -LRB- v -RRB- + d0 > = 3 then For every S -LRB- u -RRB- in S0 do Add edge uv to F . Union S -LRB- v -RRB- and S -LRB- u -RRB- . Increase d -LRB- v -RRB- by d0 . One can easily verify that the resulting F is a maximally leafy forest for G. Using the data structure in -LSB- 26 -RSB- , the first step runs in time O -LRB- -LRB- m + n -RRB- ff -LRB- m ; n -RRB- -RRB- . The second step can be done by shrinking each leafy subtree in F into a single node , and then finding a spanning tree for the corresponding shrinked graph . This can certainly be done in time O -LRB- m + n -RRB- . It follows that the time complexity of our algorithm is O -LRB- -LRB- m + n -RRB- ff -LRB- m ; n -RRB- -RRB- , which is almost linear in the size of the graph . References -LSB- 1 -RSB- S. Arora , C. Lund , R. Motwani , M. Sudan , and M. Szegedy , \ Proof verification and the hardness of approximation problems , '' Proceedings of the Thirty-third Annual IEEE Symposium on Foundations of Computer Science , -LRB- 1992 -RRB- , pp. 14 -LCB- 23 . -LSB- 2 -RSB- S. Arora , and S. Safra , \ Probabilistic checking of proofs : A new characterization of NP , '' Proceedings of the Thirty-third Annual IEEE Symposium on Foundations of Computer Science , -LRB- 1992 -RRB- , pp. 2 -LCB- 13 . -LSB- 3 -RSB- H. L. Bodlaender , \ On linear time minor tests and depth first search , '' Proceedings of Workshop on Algorithms and Data Structures , -LRB- 1989 -RRB- , pp. 577 -LCB- 590 . -LSB- 4 -RSB- G. A. Croes , \ A method for solving traveling-salesman problems , '' Operations Research 6 , -LRB- Nov.-Dec . 1958 -RRB- , pp. 791 -LCB- 812 . -LSB- 5 -RSB- E. W. Dijkstra , \ Self-stabilizing systems in spite of distributed control , '' Communications of ACM 17 , -LRB- Nov. 1974 -RRB- , pp. 643 -LCB- 644 . -LSB- 6 -RSB- T. Dimitriou , and R. Impagliazzo , \ Towards an analysis of local optimization algorithms , '' Proceedings of the Twenty-eighth Annual ACM Symposium on the Theory of Computing , -LRB- 1996 -RRB- . -LSB- 7 -RSB- M. R. Fellows and M. A. Langston , \ On well-partial-order theory and its applications to combinatorial problems of VLSI design , '' Technical Report CS-88-188 , Dept. of Computer Science , Washington State University , 1988 . -LSB- 8 -RSB- M. F?urer and B. Raghavachari , \ Approximating the minimum degree spanning tree to within one from the optimal degree , '' Proceedings of the Third Annual ACM-SIAM Symposium on Discrete Algorithms , -LRB- 1992 -RRB- , pp. 317 -LCB- 324 . -LSB- 9 -RSB- M. F?urer and B. Raghavachari , \ Approximating the minimum-degree Steiner tree to within one of optimal '' , Journal of Algorithms 17 , -LRB- 1994 -RRB- , p. 409 -LCB- 423 . -LSB- 10 -RSB- G. Galbiati , F. Maffioli and A. Morzenti , \ A short note on the approximability of the maximum leaves spanning tree problem , '' Information Processing Letters 52 , -LRB- 1994 -RRB- , pp. 45 -LCB- 49 . -LSB- 11 -RSB- M. R. Garey and D. S. Johnson , Computers and Intractability : A guide to the theory of NP - completeness , W. H. Freeman , San Francisco -LRB- 1979 -RRB- . -LSB- 12 -RSB- J. R. Griggs , D. J. Kleitman , and A. Shastri , \ Spanning trees with many leaves in cubic graphs , '' Journal of Graph Theory 13 , -LRB- 1989 -RRB- , pp. 669 -LCB- 695 . -LSB- 13 -RSB- M. M. Halld?orsson \ Approximating discrete collections via local improvements , '' Proceedings of the Sixth Annual ACM-SIAM Symposium on Discrete Algorithms , -LRB- 1995 -RRB- , pp. 160 -LCB- 169 . -LSB- 14 -RSB- D. S. Johnson , \ Approximation algorithms for combinatorial problems , '' Journal of Computer and System Sciences 37 , -LRB- 1988 -RRB- , pp . 79 -LCB- 100 . -LSB- 15 -RSB- D. S. Johnson , C. H. Papadimitriou , and M. Yannakakis , \ How easy is local search ? '' Journal of Computer and System Sciences 37 , -LRB- 1988 -RRB- , pp. 79 -LCB- 100 . -LSB- 16 -RSB- B. W. Kernighan and S. Lin , \ An efficient heuristic procedure for partitioning graphs , '' BSTJ 49 , -LRB- 1970 -RRB- , pp. 291 -LCB- 308 . -LSB- 17 -RSB- D. J. Kleitman and D. B. West , \ Spanning trees with many leaves , '' SIAM Journal on Discrete Mathematics 4 , -LRB- Feb. 1991 -RRB- , pp. 99 -LCB- 106 . -LSB- 18 -RSB- S. Lin and B. W. Kernighan , \ An effective heuristic algorithm for the Traveling-Salesman Problem , '' Operations Research 21 , -LRB- 1973 -RRB- , pp . 498 -LCB- 516 . -LSB- 19 -RSB- H.-I . Lu and R. Ravi , \ The power of local optimization : approximation algorithms for maximum-leaf spanning tree , '' Proceedings of the Thirtieth Annual Allerton Conference on Communication , Control and Computing , -LRB- Oct. 1992 -RRB- , pp. 533 -LCB- 542 . -LSB- 20 -RSB- C. H. Papadimitriou , A. A. Sch?affer and M. Yannakakis , \ On the complexity of local search -LRB- Extended Abstract -RRB- , '' Proceedings of the Twenty-second Annual ACM Symposium on the Theory of Computing , -LRB- 1990 -RRB- , pp. 438 -LCB- 445 . -LSB- 21 -RSB- C. H. Papadimitriou and K. Steiglitz , Combinatorial Optimization : Algorithms and Complexity , Prentice-Hall , Inc. -LRB- 1982 -RRB- . -LSB- 22 -RSB- C. H. Papadimitriou and M. Yannakakis , \ Optimization , Approximation , and Complexity classes , '' Proceedings of the Twentieth Annual ACM Symposium on the Theory of Computing , -LRB- 1988 -RRB- pp. 229 -LCB- 234 . -LSB- 23 -RSB- C. Payan , M. Tchuente , and N. H. Xuong , \ Arbres avec un nombres maximum de sommets pendants , '' Discrete Mathematics 49 , -LRB- 1984 -RRB- , pp . 267 -LCB- 273 . -LSB- 24 -RSB- R. Ravi , B. Raghavachari , and P. N. Klein , \ Approximation through local optimality : Designing networks with small degree , '' Proceedings , Twelfth Annual Conference on Foundations of Software Technology and Theoretical Computer Science , -LRB- Dec. 1992 -RRB- , LNCS 652 , pp. 279 -LCB- 290 . -LSB- 25 -RSB- J. A. Storer , \ Constructing full spanning trees for cubic graphs , '' Information Processing Letters 13 , -LRB- 1981 -RRB- , pp. 8 -LCB- 11 . -LSB- 26 -RSB- R. E. Tarjan , Data Structures and Network Algorithms , SIAM , Philadelphia , Pennsylvania , -LRB- 1983 -RRB- . -LSB- 27 -RSB- M. Tchuente , \ Sur l'auto-stabilisation dans un r?eseau d'odinateurs , '' R. A. I. R. O. Informatique Th?eorique 15 , -LRB- 1981 -RRB- , pp . 47 -LCB- 66 . -LSB- 28 -RSB- M. Yannakakis , \ The analysis of local search problems and their heuristics , '' STACS , -LRB- 1990 -RRB- , pp. 289 -LCB- 311 .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

combinatorial algorithms
communication network design
np-complete problems
performance ratio