Omnivariate Rule Induction Using a Novel Pairwise Statistical Test Abstract-Rule learning algorithms, for example, RIPPER, induces univariate rules, that is, a propositional condition in a rule uses only one feature. In this paper, we propose an omnivariate induction of rules where under each condition, both a univariate and a multivariate condition are trained, and the best is chosen according to a novel statistical test. This paper has three main contributions: First, we propose a novel statistical test, the combined 5 Â 2 cv t test, to compare two classifiers, which is a variant of the 5 Â 2 cv t test and give the connections to other tests as 5 Â 2 cv F test and k-fold paired t test. Second, we propose a multivariate version of RIPPER, where support vector machine with linear kernel is used to find multivariate linear conditions. Third, we propose an omnivariate version of RIPPER, where the model selection is done via the combined 5 Â 2 cv t test. Our results indicate that 1) the combined 5 Â 2 cv t test has higher power (lower type II error), lower type I error, and higher replicability compared to the 5 Â 2 cv t test, 2) omnivariate rules are better in that they choose whichever condition is more accurate, selecting the right model automatically and separately for each condition in a rule. A rule contains a conjunction of propositions and a class code that is the label assigned to an instance that is covered by the rule. An example rule containing two propositions for class C 2 is IF ðx 2 ¼ 'red'Þ AND ðx 1 < 1:5Þ THEN C 2 :. The propositions are of the form x i ¼ v, x i < or x i ! , depending on, respectively, whether the input feature x i is discrete or numeric (Rules can also be extended from propositions to the first-order case, but this is beyond the scope of this paper). A rule set is an ordered list of such rules. A test instance is classified by sequentially checking rules in a rule set. The first rule that covers the test instance assigns its class to the instance.. Rule induction algorithms have a number of desirable properties:. 1. For each class C i , there is a rule set in conjunctive normal form. Each rule set merges rules with ORs, and each rule consists of conditions merged via ANDs. Such rule sets are easy to understand, allowing knowledge extraction and validation by application experts. 2. These methods are nonparametric in that they assume no a priori form on the model or class densities and fit their complexity to that of data. 3. They learn fast and can be used on very large data sets with a large number of instances.. 4. They do their own feature extraction/dimensionality reduction and can be used on data sets with a large number of features. For an excellent review of rule induction algorithms, see [1] . Because of these reasons, rule induction algorithms are more and more frequently used in many data mining and pattern recognition applications and are preferred over other methods like artificial neural networks. Popular rule induction algorithms are C4.5RULES [2] , PART [3] , CN2 [4] , and RIPPER [5] . C4.5RULES generates a decision tree and then converts it to a set of rules by writing each path from the root to a leaf as a rule; PART grows a partial decision tree and generates a single rule from the best performing leaf; RIPPER and CN2 directly generate a set of rules. We did a survey on the separate-and-conquer rule learning algorithms in the literature, which will be given in Section 2.1.. A rule as in (1) is univariate in that conditions in the rule use only one feature. As such, each condition defines an axis-aligned split (e.g., x 1 < 1:5 versus x 1 ! 1:5) and a rule that is a conjunction of such conditions define hyperrectangles in the input space. It is obvious that if the underlying class discriminant is not axis aligned, then univariate conditions will not be appropriate. We can define multivariate conditions where several features are used to define a condition. For example, a multivariate linear condition is of the form:. for suitable values of w j ; j ¼ 0; . . . ; d. The univariate condition is a special case, where one of w j ; j ¼ 1; . . . ; d is 1, and all others are 0. This possibility of generalizing from univariate to multivariate nodes has been previously noticed for decision trees, and tree induction methods that use multivariate decision nodes have been proposed [6] , [7] , [8] , [9] . Details of these approaches will be given in Section 2.2.. Although multivariate nodes have been used in decision trees nearly for three decades, learning multivariate rules directly from data is rare. Our first contribution is the proposal of the multivariate version of RIPPER namely R.MULTI. There are three main differences between R.MULTI and RIPPER:. . R.MULTI learns rules containing only multivariate linear conditions, which are in the form of (2), whereas RIPPER just contains univariate conditions. . R.MULTI extracts multivariate linear conditions via linear support vector machine (SVM), whereas RIPPER explores the univariate feature space exhaustively. . R.MULTI uses cross validation (cv) for model selection, whereas RIPPER uses minimum description length (MDL). Our second contribution is the proposal of a new test, the combined 5 Â 2 cv t test, which is a one-sided variant of the 5 Â 2 cv t test proposed by Dietterich [10] . We also give the theoretic connections to other tests such as 5 Â 2 cv F test and k-fold paired t test. The 5 Â 2 cv t test uses only the difference of the first fold error values of the algorithms, and this is not robust due to the variance in folds. We replace this with a statistic that uses all 10 differences of error values and is, therefore, more robust. The combined 5 Â 2 cv F test uses square of the differences of error values and cannot be used as a one-sided test.. Previously, omnivariate decision trees (ODTs) have been proposed, where at each decision node, univariate and multivariate candidate nodes are trained, and the more accurate, as chosen by a statistical test, is used [11] . Our third and main contribution is the combination of the previous two contributions into a single algorithm, where we propose an omnivariate version of rule learning, R.OMNI, in which conditions may be univariate or multivariate, and the more accurate is chosen by the combined 5 Â 2 cv t test. Although one can advocate that using the combined 5 Â 2 cv F test may be enough for model selection, experimental results on 54 data sets show that R.OMNI using the proposed combined 5 Â 2 cv t test generates simpler and as accurate rule sets compared to R.OMNI that uses the combined 5 Â 2 cv F test. Fig. 1 shows example rule sets generated by R.UNI, R.MULTI, and R.OMNI algorithms for Iris data set. In this data set, R.UNI and R.MULTI generate rules containing only univariate and multivariate conditions, respectively. On the other hand, R.OMNI matches the complexity of a condition with the complexity of the subproblem defined by the data training the decision and generates two rules: one with a univariate and the other with a multivariate condition.. The paper is organized as follows: In Section 2, we review rule induction approaches. We derive our proposed combined 5 Â 2 cv t test and discuss its connections to other tests in Section 3. In Sections 4 and 5, we propose our novel multivariate R.MULTI and omnivariate R.OMNI rule induction methods. We give the experimental results in Section 6 and conclude in Section 7. In this paper, first, we propose a novel test, the combined 5 Â 2 t test, to use as a one-sided test in comparing two classifiers. Although there are statistical tests in the literature such as the 5 Â 2 t test [10] or the 5 Â 2 F test [51] , they have their own drawbacks. The 5 Â 2 t test uses only the difference of the first fold error values of the algorithms, which is not robust due to the variance in folds. We replace this with a statistic that uses all differences of error values and is, therefore, more robust. The 5 Â 2 cv F test uses square of the differences of error values and cannot be used for a one-sided test. To validate our novel test, we compare it with three well-known tests namely 5 Â 2 cv t test, 5 Â 2 F test, and kfold paired t test. We see that the one-sided combined 5 Â 2 t test has higher power (lower type II error); that is, it rejects when there is a large difference between the error values of two algorithms. Similarly, when there is not any significant difference between the error values of algorithm, the combined test accepts; that is, it has a lower probability of reject indicating that it has lower type I error. The combined test has also higher replicability; that is, tests performed by different practitioners with the same pair of algorithms, the same data sets, and the same hypothesis test present similar results.. In the second part of our work, we propose two novel rule induction algorithms R.MULTI and R.OMNI, where R.MULTI uses multivariate conditions and R.OMNI chooses between univariate and multivariate conditions using cv.. Our simulation results show that R.UNI and R.MULTI algorithms perform well on different data sets and R.OMNI performs as well as the better of them. Multivariate conditions are more complex but rules can be generated by using a smaller number of them.. As a future work, we want to combine an ensemble of multivariate rules. Boosted version of R.UNI, SLIPPER, is given in [61] . Since multivariate conditions have more variance than univariate conditions, we expect bagged or boosted R.MULTI to have smaller error rate. Fig. 9 . The number of times univariate and multivariate linear models selected in rule sets produced by R.OMNI(F) and R.OMNI(t) on 10 validation runs with respect to the level of the condition in the rule. , that is, 5 Â 2 cv t test rejects more than the combined 5 Â 2 cv t test, when A 1 is significantly larger ( ffiffiffiffiffi 10 p times) than the mean of A i s. As explained above, this makes 5 Â 2 cv t less robust than our proposed the combined 5 Â 2 cv t test since the result of the test depends on the order of folds and replications.. , that is, 5 Â 2 cv F test rejects more than the combined 5 Â 2 cv t test, when the sum of squares is larger than the square of sums. If the first classifier is better than the second in half of the folds, and if the reverse is true for the other half of the folds, F 0 a will be far larger than F 0 y , and 5 Â 2 cv F test will reject the equality of performances of those two classifiers, whereas the combined 5 Â 2 cv t test accept it.. , that is, k-fold paired t test rejects more than the combined 5 Â 2 cv t test, when the average of the products of differences in k folds (of one replication) is larger than the average of the products in two folds of the five replications. For small samples, k-fold cv will produce very small test sets, which in turn increase variance and F 0 k . Therefore, k-fold paired t test due to its nature, may be more sensitive to outliers than the combined 5 Â 2 cv t test. Although, in general, R.MULTI is more accurate than R.UNI, there are also some other data sets, where R.UNI is more accurate. This implies that it is worth using multivariate and/or rules in certain cases. We also see that R.OMNI having both univariate and multivariate conditions outperforms R.UNI on the same 12 data sets where R.MULTI also outperforms R.UNI (balance, breast, car, german, hayesroth, heart, ironosphere, optdigits, promoters, tae, vehicle, wine), and it outperforms R.MULTI on four data sets where R.UNI also outperforms R.MULTI (magic, satellite47, pageblock, ringnorm). This shows that R.OMNI uses whichever type of condition is more appropriate, selecting the right model appropriately.. The results of the Nemenyi's test on the number of rules and number of conditions are given in Figs. 8b and 8c , respectively. With respect to the rule set complexity, in terms of number of rules, there are four cliques: (R.OMNI(t), R.OMNI(F), R.MULTI), (R.OMNI(F), R.UNI), (R.UNI, LDT, CART), and (CN2, C45, PART). In terms of number of conditions, there are five cliques: (R.OMNI(t), R.OMNI(F), R.MULTI), (R.OMNI(F), R.UNI), (R.UNI, LDT, CART), (CART, CN2), and (CN2, C45, PART). This is mainly due to the fact that multivariate conditions are more complex than univariate conditions, and one multivariate condition is equivalent to multiple univariate conditions. Usually, R.MULTI generates significantly smaller rules on data sets with discrete attributes. For example, on balance, R.UNI generates on the average 4.0 rules whereas R.MULTI generates 1.5; on mushroom, R.UNI generates 8.1 rules compared to 1.1 rules of R.MULTI; and on tictactoe, R.UNI generates 8.2 rules compared to R.MULTI's 1.0.. Since R.OMNI uses a combined 5 Â 2 cv F or t test, it requires 10 runs of univariate and multivariate condition optimizations. Therefore, it is the slowest of the three algorithms. R.UNI algorithm requires OðdNlogNÞ (sorting N data for d features using Quicksort) to find the best condition, whereas R.MULTI requires OðN 3 Þ (solving convex optimization problem) to find the best condition. Therefore, if d is significantly larger than N, R.MULTI is faster than R.UNI; otherwise, R.UNI is faster than R.MULTI. The result of Nemenyi's test on the training time of the rule sets produced by C45, CN2, PART, CART, LDT, R.UNI, R.MULTI, R.OMNI(F), and R.OMNI(t) algorithms is shown in Fig. 8d . Now, there are six cliques: starting with the fastest univariate learners (C45, CN2, R.UNI), (CN2, R.UNI, PART), (PART, LDT), continuing with the multivariate learners (LDT, R.MULTI), (R.MULTI, R.OMNI(F), R.OMNI(t)), and the slowest omnivariate learners with CART (R.OMNI(F), R.OMNI(t), CART).. The number of times univariate and multivariate linear models selected in rule sets produced by omnivariate algorithms R.OMNI(F) and R.OMNI(t) are given in Table 2 . Also, the selection counts of those models as a function of the position of a condition in a rule for R.OMNI(F) and R.OMNI(t) algorithms are given in Fig. 9 . In R.OMNI(F), linear models are selected less than univariate models, whereas in R.OMNI(t) univariate models are selected less than the multivariate linear models. In 41 data sets out of 54 data sets, a linear model is chosen more than the univariate model by R.OMNI(t). On the other hand, in 28 data sets out of 54 data sets, a univariate model is chosen more than the linear model by R.OMNI(F).. The difference is mainly due to the nature of two statistical tests. In Section 6.1.3, we showed that combined 5 Â 2 cv F test is more conservative than the combined 5 Â 2 cv t test. That is, combined 5 Â 2 cv t test rejects the null hypothesis that univariate model and multivariate model have the same expected error more than the combined 5 Â 2 cv F test. If the null hypothesis is rejected more, since usually the linear model is better than the univariate model, the linear model is favored more. On the other hand, if the null hypothesis is not rejected (like usually in 5 Â 2 cv F test), the simple model, namely univariate model, is favored more.. If we look at Fig. 9 , we see that multivariate conditions are usually selected early on, closer to the beginning of the rule, where the complexity of the problem is high. We see that when there is few data, it is more likely that the appropriate condition is univariate, whereas with more data, a multivariate condition becomes more likely. When learning a rule, we add one condition at a time, and each condition covers a number of instances that are removed from the data. While learning the initial conditions (when there are more data), it is more likely to have a multivariate condition; as we proceed down the rule, we have less remaining data and the condition is more likely to be univariate-the multivariate condition overfit and is rejected by the statistical test. The seven classification algorithms we use because of their low time/space complexity are as follows:. 1. max decides based on the prior class probability without looking at the input. All test instances are assigned to the class with the maximum prior. It is not a learning algorithm in the usual sense, but any plausible learning algorithm must have smaller error rate than max; it is indeed surprising that max is sometimes quite accurate. 2. mean is the nearest mean classifier that keeps the mean vector for each class and assigns instance to the class whose mean has the smallest euclidean distance to the instance [54] . This corresponds to assuming that classes are Gaussian distributed with a shared covariance matrix whose diagonals are equal and off diagonals are 0. 3. lda is the well-known linear classification algorithm. 4. lnp linear perceptron with softmax outputs trained by gradient-descent to minimize cross entropy. 5. RIPPER. 6. c45 is the archetypal decision tree method [2] . 7. nn [55] is the 1-nearest neighbor classification algorithm and uses the euclidean distance. These classification algorithms are tested on 40 data sets from the UCI machine learning repository [56] . Each data set is resampled using 5 Â 2 cv [10] where twofold cv is done five times (with stratification) and the roles swapped at each fold to generate 10 training and testing folds.. We use a total of 54 data sets from UCI [56] and Delve [58] repositories. Our methodology in the division of training, validation, and test sets is as follows: A data set is first divided into two parts, with 1=3 as the test set, test, and 2=3 as the training set, train-all. The training set, train-all, is then resampled using 2 Â 5 cv where fivefold cv is done two times (with stratification) and the roles swapped at each fold to generate 10 training and validation folds, tra i ; val i ; i ¼ 1; . . . ; 10. tra i are used to learn the rule sets. The optimization and simplification of the rule sets are done using val i . The test set, test, is used to estimate the generalization error of the rule sets. We compare nine algorithms in the second part of our experiments:. 1. C45: C4.5 is the archetypal decision tree method [2] . 2. CN2: CN2 rule learning algorithm. 3. PART: PART rule learning algorithm. 4. CART: Multivariate decision tree learning algorithm using backfitting to find the discriminant at each decision node. 5. LDT: Multivariate decision tree learning algorithm using LDA to find the discriminant at each decision node.. 6. R.UNI: Ripper the proper, original univariate rule induction algorithm. 7. R.MULTI: Our proposed multivariate rule induction algorithm, where an SVM with linear kernel is used to find multivariate linear conditions. We use the LIBSVM 2.82 library that implements the linear SVM [59] . We tune the regularization parameter C using val i on each data set. 8. R.OMNI(F): Our proposed omnivariate rule induction algorithm, where combined 5 Â 2 cv F test is used to compare univariate and multivariate candidate models for each decision condition. 9. R.OMNI(t): Our proposed omnivariate rule induction algorithm, where combined 5 Â 2 cv t test is used to compare univariate and multivariate candidate models for each decision condition. In our simulations, we saw that the optimization phase should be repeated twice, also for the multivariate case, as in RIPPER. These two optimization iterations do decrease error rate and rule set size, but repeating it further does not have such a drastic effect and does not justify the extra training time.. Our comparison criteria are the generalization error of the algorithms, the number of rules and the number of conditions in those rules they generate, and the time required to train the algorithm.. Since we have more than two algorithms, on each data set, we cannot compare the average performances on 54 data sets and one needs to resort to nonparametric tests. Friedman's test and its posthoc Nemenyi's test use ranks instead of the absolute performances [60] . On each data set, the performance values of the algorithms are sorted from the best to the worst so that the best one gets the rank of 1, the second 2, and so on, until we get to 9. We then use nonparametric tests to check for significant difference in average ranks over the 54 data sets.. According to Nemenyi's test, two neighboring algorithms lead to classifiers with significantly different performance ranks at significance level if the difference of their average ranks is greater than or equal to the critical difference. where L is the number of algorithms, M is the number data sets, and q is the Studentized range statistic divided by ffiffi ffi 2 p . This allows us to find cliques of equally good subsets that we can represent by underlining them. Table 1 shows the average and standard deviations of expected error of rule sets produced by C45, CN2, PART, CART, LDT, R.UNI, R.MULTI, R.OMNI(F), and R.OMNI(t) algorithms. With respect to the expected error, R.MULTI is better than both omnivariate algorithms that seem to be similar and are better than R.UNI proper. The results of the Nemenyi's test on the expected error is given in Fig. 8a . We see that there are three cliques: (R.OMNI(t), R.OMNI(F), R.MULTI, LDT), (R.OMNI(t), R.OMNI(F), LDT, PART, R.UNI, CART, C45), and (PART, CART, R.UNI, CN2, C45). As we have explained in Section 4, a linear SVM is a powerful classifier, so R.MULTI is better than R.UNI in many data sets, with significant differences occurring especially on discrete data sets such as balance, car, hayesroth, and promoters. Some data sets are also inherently complex, and in those data sets, again R.MULTI is significantly better than R.UNI such as dermatology, ocr, optdigits, and tae. Contrarily, on the handcrafted data set monks, R.UNI can learn the optimal rules and is significantly better than R.MULTI, which overfits for this case.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Drawing trees in a streaming model Abstract. We introduce a data stream model of computation for Graph Drawing, where a source produces a graph one edge at a time. When an edge is produced, it is immediately drawn and its drawing can not be altered. The drawing has an image persistence, that controls the lifetime of edges. If the persistence is k, an edge remains in the drawing for the time spent by the source to generate k edges, then it fades away. In this model we study the area requirement of planar straight-line grid drawings of trees, with different streaming orders, layout models, and quality criteria. We assess the output quality of the presented algorithms by computing the competitive ratio with respect to the best known offline algorithms. We consider the following model. A source produces a graph one edge at a time. When an edge is produced, it is immediately drawn (i.e., before the next edge is produced) and its drawing can not be altered. The drawing has an image persistence, that controls the lifetime of edges. If the persistence is infinite, edges are never removed from the drawing. Otherwise, suppose the persistence is k, an edge remains in the drawing for the time spent by the source to generate k edges, and then it fades away.. Studying this model, which we call streamed graph drawing, is motivated by the challenge of offering visualization facilities to streaming applications, where massive amounts of data, too large even to be stored, are produced and processed at a very high rate [12] . The data are available one element at a time and need to be processed quickly and with limited resources. Examples of application fields include computer network traffic analysis, logging of security data, stock exchange quotes' correlation, etc.. For the user of the visualization facility it is natural to associate any graphic change with a new datum coming from the stream. Hence, moving pieces of the drawing would be potentially ambiguous. On the other hand, the drawing should have a size as limited as possible.. Although streamed graph drawing is related to incremental and dynamic graph drawing, it is qualitatively different from both. In incremental graph drawing the layout is constructed step by step according to a precomputed vertex ordering that ensures invariants regarding, e.g., its shape [3, 7] . In streamed graph drawing the order cannot be chosen. Dynamic graph drawing [4, 11, 13] usually refers to drawing sequences of graphs, where drawings of consecutive graphs should be similar. Insertions and/or deletions of vertices/edges are allowed and the current graph must be drawn without knowledge of future updates. However, the current layout is only weakly constrained by previous drawings. In streamed graph drawing modifications concern only single edges and previous layout decisions may not be altered.. While there is some work on computing properties of streamed graphs (see, e.g., [1, 5, 8] ), very little has been done in the context of graph drawing. A result that applies to streamed graph drawing with infinite persistence is shown in [13] in what is called no change scenario. In that paper, a graph of maximum degree four is available onevertex-at-time and it is drawn orthogonally and with a few crossings.. We consider both a finite persistence and an infinite persistence model. Our results in these models concern the area requirement for planar straight-line grid drawings of trees, where we assume that the tree is streamed in such a way that the subtree to be drawn is connected. Since a streamed graph drawing algorithm is a special case of an online algorithm, it is reasonable to assess its output quality in terms of its competitive ratio with respect to the best known offline algorithm. The obtained results are summarized in Table 1 , where n is the number of vertices of the current graph. For each type of streaming order and for each class of trees investigated within each model, the table reports the competitive ratio of a (specific) drawing algorithm, and the corresponding lemma/theorem. The table puts in evidence the practical applicability of the finite persistence model. On the other hand, the results on the infinite persistence model show the intrinsic difficulty of the problem. In fact, in the paper we prove that a large family of algorithms for the infinite persistence model requires Ω(2 n 8 /n) competitive ratio to draw binary trees (see Lemma 5) .. Another way to interpret our results on the infinite persistence model is the following: All the area-efficient tree-drawing algorithms known in the literature have the capability to inspect the entire tree for exploiting some balancing consideration. In the infinite persistence model we ask the question of which is the achievable area bound if such an inspection can not be done.. This paper is organized as follows. In Sect. 2 we introduce the concept of streamed graph drawing. Area requirements for tree drawings in our two main models are derived in Sects. 3 and 4, and we conclude with directions for future work in Sect. 5.

O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Stability of a coupling technique for partitioned solvers in FSI applications This paper focuses on the stability of the coupling iterations in the partitioned approach to fluid-structure interaction. Previous research has shown that the number of coupling iterations increases when the time step decreases or when the structure becomes more flexible which is explained here by Fourier error analysis of the unsteady, incompressible flow in an elastic tube. Substituting a linearized model of the structural solver into the flow solver makes the coupling more stable but is impracticable if the flow solver is a black box. Therefore the coupling iterations are stabilized by coupling with reduced-order models and Aitken underrelaxation. Recently fluid-structure interaction has received a lot of attention in several domains. In aeronautics [1] and turbomachinery [2] , fluid-structure interaction is used for flutter analysis. Parachute dynamics is another interesting field as large deformations appear [3] . Biomedical applications (e.g. heart valves [4] , arteries [5, 6] ) are challenging as the interaction between the fluid and structure is strong due to the flexible structure and the comparable density of the fluid and the structure.. Fluid-structure interaction and other coupled problems can be dealt with using a monolithic or a partitioned approach. In the monolithic approach, both the flow and structure problem are solved with a single code [7] [8] [9] [10] [11] . This avoids the coupling iterations to find the coupled solution in the partitioned approach where the flow and structure are solved with separate codes. On the other hand, the monolithic approach nowadays requires the use of a dedicated code [7] , whereas the partitioned approach allows the reuse of existing codes as long as the solvers for flow and structure are treated as black boxes. Only partitioned approaches to fluid-structure interaction are considered here.. Several coupling schemes for the partitioned approach have been devised. In a staggered scheme, the flow and structure problem are solved sequentially and no coupling iterations are performed within a time step [12] [13] [14] . This scheme is only stable in cases with weak interaction between the fluid and the solid, like aeroelastic applications.. Before introducing coupling schemes that do require coupling iterations, the flow solver (F or f ) and the structural solver (S or s) are written in fixed point formulation. and root-finding formulation. with X the position of the fluid-structure boundary and Y the fluid load on it. The fluid-structure interaction problem in the latter formulation can be solved with Newton-Raphson iterations [15, 16] . If the linear systems are solved with a direct method, this approach requires knowledge of the Jacobian of the equations (2), which can be very time consuming or difficult to calculate for black box solvers. Therefore also quasi-Newton methods with approximate Jacobians are used [17] [18] [19] .. In the fixed point formulation, the coupled solution can be found with coupling iterations between the flow and structure problem. However, this is prone to instability when the density ratio of fluid and structure increases and when the structure becomes more flexible [20, 21] . In previous research [5, 22, 23] with a very flexible, light or zero-mass structure and an incompressible fluid, it is. shown that the number of coupling iterations increases when the time step decreases. So unlike many other situations, decreasing the time step does not facilitate the solution of the problem [24, 25] . It is important to understand this tendency in order to improve coupling techniques for the partitioned solution of coupled problems.. There exist several techniques to stabilize the coupling iterations between a black box fluid and structural solver, like Aitken's method [26] , which uses a variable underrelaxation of the fixed point iterations. A scheme to couple two black box solvers by making linear reduced-order models (ROMs) of those solvers has been introduced by Vierendeels et al. [5] . Aitken underrelaxation and coupling with reduced-order models will be discussed further in this paper.. Notwithstanding the focus of this paper on coupling techniques which do not require modification of the partitioned solvers, a lot of insight can be gained. by substituting a linearized model of the structure into the flow solver. The substitution terms in the flow equations can be activated or deactivated, so coupling schemes ranging from monolithic to completely partitioned can be analyzed.. To enable both analytical and numerical analysis, a one-dimensional problem is studied namely the unsteady flow in an elastic tube. This is a simplified model of the pulsating blood flow in an artery for which two-dimensional simulations have been presented in previous work [5] . Although a one-dimensional model does not accurately represent the complex phenomena in an artery, its behaviour from a fluid-structure point of view is very similar. A onedimensional model is outstandingly suited to analyze the effect of the time step, space step and elasticity of the structure on the number of coupling iterations. Fourier error analysis is used to investigate the stability of the coupling iterations between flow and structure for several coupling schemes.. The remainder of this paper is organized as follows. A detailed description of the flow and structure problem is given in section 2 and a simple fixed point iterative coupling scheme is introduced in section 3. The Fourier error analysis of this scheme follows in section 4 and the results of the error analysis are verified in section 5 by means of simulations with a linear structure model.. In section 6 the coupling scheme with reduced-order models and Aitken underrelaxation are outlined and in section 7 the performance of coupling with reduced-order models is compared with fixed point iterative coupling with and without Aitken underrelaxation using a non-linear structure model.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Precise eye localization using HOG descriptors In this paper, we present a novel algorithm for precise eye detection. First, a couple of AdaBoost classifiers trained with Haar-like features are used to preselect possible eye locations. Then, a Support Vector Machine machine that uses Histograms of Oriented Gradients descriptors is used to obtain the best pair of eyes among all possible combinations of preselected eyes. Finally, we compare the eye detection results with three state-of-the-art works and a commercial software. The results show that our algorithm achieves the highest accuracy on the FERET and FRGCv1 databases, which is the most complete comparative presented so far. Eye detection is the natural link between face detection and recognition algorithms. Surprisingly, most face recognition approaches assume that faces are precisely located and most holistic face detection algorithms ignore the issue of precision. Precision is important for face recognition as it has been proved in some recent works [1] [2] [3] , which show that for most algorithms the recognition rate decreases rapidly as the eyes shift from the ideal locations. For this reason, many authors consider today that the problem of precise face localization is of great importance. In the last years, two types of images have been used for the location of eyes, namely IR and RGB images. In the case of IR images, the key idea is to detect eyes looking for the reflection of the pupils under near IR illumination [4, 5] . However, these methods require a specific technology and equipment that is not always available. Methods that use RGB images can be broadly grouped into three categories: template matching, feature and appearance based. In template algorithms [6, 7] , several standard patterns of an eye are used for searching using cross-correlation. These algorithms are computationally expensive as the number of operations they require is proportional to the size and number of templates, usually high. Feature-based algorithms [7] [8] [9] [10] use previous knowledge to define a set of heuristic rules. For instance, in [8] the author exploits the facts that the eyes have strong horizontal edges and that the pupil is darker than the surrounding facial regions, while in [9] they use the information of the edges and the intensity to locate the iris. However, the a priori knowledge used in feature-based algorithms is not always easy to determine. Also, most approaches cannot cope with a large number of environments for eye detection as it is difficult to find features capable to adapt to different conditions. Finally, appearance-based algorithms use supervised classifiers which are fed with low level features. Some examples of low level features include wavelets [11, 12] , Gabor features [13, 14] and Haar-like features [15, 16] , among others.. Once features are extracted supervised classification algorithms are used to differentiate between eyes and non-eyes. For instance, in [17] a Support Vector Machine (SVM) classifier [18] which integrates the characteristics of single eye and eye-pair images is used to develop a hybrid classifier. The features that describe the eyes are selected via a filter and a wrapper method for a simplified maximum likelihood, and verified with template matching scores to reject false detections. Support machines are also proposed in [12] ; in this work, the SVM is trained on properly selected Haar wavelet coefficients. The AdaBoost classifier [19] has also been extensively used for the classification of eyes [3, 16, 20] . In [3] , AdaBoost is used with features extracted after a Recursive Nonparametric Discriminant Analysis (RNDA); in [20] , the authors model images with a generative approach that uses a boosting stage to learn these models. In [16] , AdaBoost is used in a preliminary coarse eye detection stage. This idea is also used in our work as it will be shown in Sect. 2.. Other solutions have also been proposed for precise face and eye localization. For instance, in [21] , a genetic algorithm combined with decision trees is used to classify salient locations as eye regions. In [22] , eye localization is achieved using hierarchical neural networks with multi-resolution and local recurrent connectivity.. Although some good performances are reported in the previous approaches, still further efforts are needed to increase the precision/recall of eye detection systems to improve the recognition rates of fully automatic face recognition systems. In this paper, we present a novel and accurate eye localization approach. The novelty of our approach lies in the use of a SVM classifier trained with Histograms of Oriented Gradients (HOG) features [23] obtained after normalization of the face. HOG features, which have been previously used in applications such as pedestrian detection [23] and face recognition [24] , are closely related to the Scale Invariant Feature Transform (SIFT) [25] . SIFT features have emerged as a cutting edge technology for extracting distinctive features from images, to be used in algorithms for tasks like matching different views of an object or scene and face authentication [26] . One of the essential parts of SIFT is that local keypoints are represented using histograms of image gradients which are normalized to be invariant to changes in scale and rotation. While these normalizations have proved to be very useful for images that are arbitrarily scaled or rotated, the fact is that they remove information which might be useful for recognition when images are not scaled or rotated. HOG features, similar to SIFT, also use gradient to describe local keypoints. The main difference to SIFT features is that scale and rotation normalization stages are skipped, which is an advantage when they are extracted from normalized faces. Another important difference is that SIFT features are extracted at keypoints which are local extrema of the spacescale representation of the image, while our HOG features are extracted at selected keypoints as will be described below. Figure 1 shows the block diagram of our approach. The input to the system is a bounding box that contains a frontal face. The reader can see [27] for a complete survey on face detection approaches although we recommend the algorithm proposed by Viola and Jones in [19] .. Similar to most object localization approaches, we perform eye detection by scanning a window at different scales over the image. This exhaustive scanning allows to deal over variations of position and scale. In our approach, we first scan the upper part of the face region with an AdaBoost classifier [19] that uses Haar-like features. The goal of this step is to preselect eye candidates at different scales with high recall and reject most negative subwindows efficiently. Note that this step is computed over a small region of the face which allows to keep the computational burden low as it will be shown in Sect. 6. Next for each possible pair of eyes candidates, a normalized face is extracted by placing the eyes at fixed coordinates and thus compensating for rotation and scale changes. Finally, SVM is used to select the best candidate using HOG descriptors obtained from the eyes of the normalized faces.. The rest of the paper is organized as follows. Sections 2 through 4 explain in detail each block of the diagram presented in Fig. 1 . The methodology and the datasets used to test the system are described in Sect. 5 and the results exposed in Sect. 6. Finally, a summary and conclusions are drawn in Sect. 7. In this paper, we presented a new method to precisely localize eyes. The method consists of two major steps. First, we detect efficiently a set of possible eye locations using a couple of Haar-like AdaBoost classifiers. Then, for each possible eye pair, we extract and normalize a face. Using the normalized faces, we select the best eye pair using a SVM classifier trained with HOG descriptors. The test experiments have been carried out on the FERET and FRGCv1 databases. Using the same data and methodology, we have compared the results with other recent state-of-the-art approaches. The results show that our algorithm achieves the highest precision on both databases. Finally, we would like to emphasize that although our implementation pipelines several stages, the overall computational complexity is similar to that of other state-of-the-art algorithms. In this section, we provide details about the data used to train and test our system and about the methodology followed to train the classifiers.

O O O O B O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Linear Circuits, Two-Variable Logic and Weakly Blocked Monoids Abstract. Following recent works connecting two-variable logic to circuits and monoids, we establish, for numerical predicate sets P satisfying a certain closure property, a one-to-one correspondence between F O[<, P]-uniform linear circuits, two-variable formulas with P predicates, and weak block products of monoids. In particular, we consider the case of linear TC 0 , majority quantifiers, and finitely typed monoids. This correspondence will hold for any numerical predicate set which is F O[<]-closed and whose predicates do not depend on the input length. The computational power of boolean circuits are of great interest as they are among the most powerful classes of computation devices for which we can prove nontrivial lower bounds [7] . To understand the power granted by nonuniformity in this setting, we often consider circuit families which can be generated under bounded resources.. In the case of small depth circuits, we are particularly interested in circuit families whose structure can be described in terms of some restricted class of logical formulae (Barrington, Immerman, and Straubing [2] ). Such circuit families can often be characterized in terms of logical formulas. For instance, the languages recog- The class of languages recognized by logical formula can be also be characterized in terms of algebra. For instance, the class of languages recognized by F O[<] formula corresponds exactly to the class of languages recognized by star-free languages, which are exactly those which are recognized via morphisms to finite aperiodic monoids, or equivalently, block products of U 1 (list some more relevant logic to algebra connections). This gives us a three-fold connection between circuits, logic and algebra.. In the case of AC 0 and ACC 0 , restricting to linear size corresponds in logic to a restriction to using only two variables. This was shown by Lautemann et al. [9] , and corresponds in algebra to weakly-blocked monoids. In [21] Therién and Wilke gave for first order formulas over two variables with the order predicate an algebraic characterization as the variety DA. By an result of Straubing and Therién [20] this variety, and thus F O 2 [<], can be characterized as weakly blocked monoids of U 1 . Analogously, F O + M OD 2 [<] was shown in [19] to correspond to the variety DO G sol , which is the closure of DA under weak block products of abelian groups.. The notion of finitely typed monoids was introduced in [8] to obtain an algebraic characterization for TC 0 = L(M AJ[<, Sq]) in terms of finitely typed monoids. It is clear that general numerical predicates, as well as linear TC 0 , need the use of infinite monoids. In this paper, we show that types can be used to algebraically characterize logical formulas for many types predicate sets in a uniform way. Second, we apply these results to give matching logical and algebraic characterizations to a broad class of uniformity conditions for linear TC 0 , which include the F O[<]-closure of the predicate sets {<}, {<, +}, and {<, arb}.. In particular we show, subject to a closure property of P, that the following properties of a language L are equivalent: (1) that it is recognized by a F O[<, P]-uniform family of T C 0 circuits of linear size and linear fan-in, (2) that it can be described by a F O + M AJ 2 [<, P] formula, and (3) that it is recognized by a restricted type of morphism into a particular type of finitely typed group, constructed from weak block products of simpler groups. Recent results suggest that this characterizations can be used to prove lower bounds on linear sized circuits [5] .. The remainder of the paper is structured as follows. In Sections 2 we review circuit and logic and in Section 3 give the algebra definitions that we will require in the exposition. In Section 4 we state the main result of this paper, and in the remaining sections we prove threewe review the circuit, logic, inclusions that give us the result. In this paper we extend the known connections between linear circuits, two-variable logic, and weakly blocked algebra from the case of linear AC 0 and linear ACC 0 to the case of linear TC 0 . This algebraic characterization can be used to prove that the word problem over A 5 (known to be complete for N C. ) was linked to weakly blocked U 1 (resp. Z p ) but no connection to circuits is known. On the other hand F O 2 [arb] (resp. F O + M OD 2 [arb]) corresponds to linear AC 0 (resp. linear ACC 0 ). We obtain a three-way correspondence for predicate sets respecting certain closure properties. Our proofs also hold for the case of F O 2 and F O+M OD 2 : The group (Z, Z + ), which simulates the quantifier, can be substituted by U 1 , or by U 1 and Z p to get results for those cases. In this way we obtain the possibility to handle predicate sets between the order predicate and arbitrary numerical predicates, e.g. {<, +}, {<, +, * }.. We want to thank Klaus-Jörn Lange and Stephanie Reifferscheid for helpful comments.. A Proofs Theorem 1. For each binary numerical predicate P (x, y) there exists a finitely typed group (T, T) and a distinguished element m ∈ T with the following properties:. 1. there is a morphism h : ({a} × 2 {x,y} ) * → T with h((a, ∅)) = m and an extended type T such that a n x=i,y=j |= P (x, y) if and only if h(a n x=i,y=j ) ∈ T . 2. for all extended types T ∈ T and all morphisms h : ({a} × 2 {x,y} ) * → T with h((a, ∅)) = m the predicate corresponding to the language h −1 (T )∩{a} * ⊗{x, y} is in {P }.. We call m the incremental element.. Proof. We can associate to each binary predicate P the subset P ⊆ Z 3 where (i, j, n) ∈ P iff a n x=i,y=j |= P . We choose T = (Z 3 , {P, Z\P}) ((Z, Z + )×(Z, Z + )). We first show the existence of an appropriate morphism. The suitable morphism h we construct will satisfy h(a, ∅) = (f ∅ , (0, 0)), h(a, {x}) = (f x , (1, 0) ), h(a, {y}) = (f y , (0, 1) ), h(a, {x, y}) = (f xy , (1, 0) ), so the position of the variables x and y will be encoded in the second parameter. It remains to define m and the f S functions.. In the following we use 1 1 1 and 0 0 0 to denote the constant 1 and 0 function respectively. Define g x : Z 2 × Z 2 → Z by:. Likewise define g y by inverting the role of the x and y position. Lastly we define:. and set m = (f ∅ , (0, 0)). Consider the value of h(w) on a word w ∈ a n ⊗ 2 x,y . Recalling equation (*), it is easy to check that π 1 (h(a n ))(e, e) = (i, j, n), where i and j are the positions of x and y respectively.. On the other hand, let h be an arbitrary restricted morphism defined by h((a, 0)) = m = (f, (0, 0)), h((a, {x})) = (f x , m x ), h((a, {y})) = (f y , m y ), h((a, {x, y})) = (f xy , m xy ). The value of π 1 (h(a n ))(e, e) falls into a finite number cases, depending on the position of x relative to y and the values of m x , m y , and m xy . For instance, consider π 1 (h(a n x=i,y=j )) for i < j. The sum in the equation (*) evaluates to:. Observe that m x f m y (e, e) has nonzero value if and only if m x = (1, 0) and m y = (0, 1), or vice versa. Likewise (0, 0)f (m x + m y )(e, e) has a nonzero value if and only if m x + m y evaluates to (1, 0), (0, 0), (0, 1) or (1, 1) . This finite set of cases can be checked by a boolean combination of queries of {P }, and so by definition the predicate evaluated by h is in {P }. The following theorem translates the well known connections between two-variable logic, weak blocked monoids, and linear size circuits [21, 19, 9] to the case of majority. By establishing a similar uniformity result as in [6] , we can show how the predicates used in logic have their counterparts in algebra and circuits in terms of uniformity.. Theorem 2. Let P be a set of predicates closed under F O[<] constructions and shifting. The following are equivalent:. Proof. First we show that we can express a circuit family by a logic formula (Theorem 4). Then we show that a language in this logic can by recognized by a restricted morphism (Theorem 5). Finally, we show how to construct a circuit family for a restricted morphism (Theorem 6).. It is unknown whether T C 0 is a hierarchy in the depth of the circuits. The following theorem connects the circuit depth of F O[<, P]-uniform LT C 0 to the quantifier depth of F O + M AJ 2 [<, P].

O O B I O B I I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Column Generation Approach to Capacitated p-Median Problems Scope and purpose The search for p-median vertices on a network (graph) is a classical location problem. The objective is to locate p facilities (medians) so as to minimize the sum of the distances from each demand vertex to its nearest facility. The capacitated p-median problem (CPMP) considers capacities for the service to be given by each median. The total service demanded by vertices identified by p-median clusters cannot exceed their service capacity.. In general, the CPMP has not seem as intensively studied as the classical p-median problem. Related problems have appeared in Bramel and Simchi-Levi [3] , Klein and Aronson [18] , Mulvey and Beck [29] and Osman and Christofides [32] . An extensive bibliography of related problems, and also a set of test problems are presented in [32] . The CPMP is known to be NP-hard. Some earlier approaches applying Lagrangean heuristics to the CPMP are proposed in Koskosidis and Powell [19] and in [29] . Recent approaches apply metaheuristics, such as simulated annealing and tabu search (as in França, Sosa and Pureza [13] and in [32] ), and genetic algorithms (Maniezzo, Mingozzi and Baldacci [25] and Lorena and Furtado [20] ). Good results are reported for a set of standard test problems (OR-Library -http://mscmga.ms.ic.ac.uk/info.html [2] ).. The Lagrangean/surrogate relaxation has been used recently to accelerate subgradient like methods, which are often used to optimize the corresponding Lagrangean dual problem as in Lorena and Lopes [21] , Lorena and Narciso [22] , Lorena and Senne [23, 24] , Narciso and Lorena [30] , and Senne and Lorena [34] . Lorena and Senne [24] explored the Lagrangean/surrogate relaxation combination with location-allocation heuristics, proposed by Cooper [5] and used before by Senne and Lorena [34] .. Column generation is a powerful tool for solving large-scale linear programming problems.. Such linear programming may arise when the columns in the problem are not known in advance and a complete enumeration of all columns is not an option, or the problem is rewritten using Dantzig-Wolfe decomposition (the columns correspond to all extreme points of a certain constraint set) [6] . It appears that this approach was not tried before for the CPMP, but has been successfully explored in several other applications, such as the well-known cutting-stock problem, vehicle routing and crew scheduling [1, 4, 6, 7, 8, 9, 14, 15, 35, 36] . However, in many cases a straightforward application of column generation may result in slow convergence. The Lagrangean/surrogate relaxation is showed in this paper to be an acceleration process to the column generation, generating new productive sets of columns.. In this paper we examine a column generation approach to the CPMP. The identified restricted master problem optimizes the covering of 1-median clusters satisfying a set of capacity constraints, and new columns are generated solving capacitated subproblems, which consider the restricted master dual variables and the clusters capacities. In this work the Lagrangean/surrogate relaxation is directly identified from the master problem dual and provides new bounds and new productive columns through a modified knapsack subproblem. The overall column generation process is accelerated, even when multiple pricing is observed. Computational tests are presented using instances taken from the literature.. The paper is organized as follows. Section two presents CPMP formulations used in this paper. The next section presents the Lagrangean/surrogate relaxation and the column generation approach to the CPMP, comparing the traditional process and the one improved by Lagrangean/surrogate multipliers. Section three presents the basic algorithms for the column generation, including the initial pool of columns and the management of columns.. Section four provides computational results to illustrate the benefits of the new approach. This paper presented column generation approaches for a CPMP. The approaches integrate the traditional column generation to the Lagrangean/surrogate relaxation context, identifying new productive columns and accelerating the computational process.. The computational results show that the Lagrangean/surrogate sub-problem generates a small number of productive columns and the restricted master is also manageable with a small number of columns.. The Lagrangean/surrogate lower bounds can be useful to branch-and-price trees and are currently being explored in this context. Columns Generated = number of columns generated; Time = total computational time (in seconds). Table 2 -Results for CG (1) The columns contain: Columns Generated = number of columns generated; Time = total computational time (in seconds). Table 3 -Results for LSLSH(t)

O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Long Range Dependence in Network Traffic and the Closed Loop Behaviour of Buffers Under Adaptive Window Control We consider an Internet link carrying http-like traffic, i.e., transfers of finite volume files arriving at random time instants. These file transfers are controlled by an adaptive window protocol (AWP); an example of such a protocol is TCP.We provide analysis for the auto-covariance function of the AWP controlled traffic into the link's buffer; this traffic, in general, cannot be represented by an on-off process. The analysis establishes that, for TCP controlled transfer of Pareto distributed file sizes with infinite second moment, the traffic into the link buffer is long range dependent (LRD).We also develop an analysis for obtaining the stationary distribution of the link buffer occupancy under an AWP controlled transfer of files sampled from some distribution. For any AWP, the analysis provides us with the LaplaceStieltjes transform (LST) of the distribution of the link buffer occupancy process in terms of the functions defining the AWP and the file size distribution. The analysis also provides a necessary and a sufficient condition for the finiteness of the mean link buffer content; these conditions again have explicit dependence on the AWP used and the file size distribution. One of the consequences of this result is to establish the sensitivity of the buffer occupancy process to the file size distribution.Combining the results from the above analyses, we provide various examples in which the closed loop control of an AWP results in finite mean link buffer occupancy even though the file sizes are Pareto distributed (with infinite second moment), and even though the traffic into the link buffer is long range dependent (with Hurst parameters which would suggest an infinite mean queue occupancy under open loop analysis).We also study certain models of window reductions due to constant rate packet marking and find that window reductions lead to further lightening of the tail of the buffer occupancy distribution.The significance of this work is threefold: (i) by looking at the window evolution as a function of the amount of data served and not as a function of time, this work provides a new framework for analysing various processes related to the link buffer under AWP controlled transfer of files with a general file size distribution; (ii) it indicates that the buffer behaviour in the Internet may not be as poor as predicted from an open loop analysis of a queue fed with LRD traffic; and (iii) it shows that the buffer behaviour is sensitive to the distribution of file sizes . It was observed in [1] that traffic processes in the Internet display long range dependence. In [2] this phenomenon was traced to the fact that the traffic in the Internet results from the transfer of files that have a heavy tailed distribution. Models have shown that the transfer of Pareto distributed files (P {V > x} = min(1, 1 x α ), 1 < α < 2) results in a traffic rate process that has an auto-correlation function that decays as 1 τ α−1 . These observations have been taken to indicate that the buffer occupancy distribution in router buffers will have heavy tails [3] . Such observations are, however, based on an "open loop" analysis of an Fig. 1 . TCP controlled file transfers over a single link connecting servers and clients; the link propagation delay is assumed to be negligible, and the link buffer is infinite. In this work we analyse the a(t) process, and the link buffer occupancy process.. input process to the link buffer depends on the feedback control used and hence it is interesting to study the correlation structure of the data arrival rate process into the link buffer (denoted by a(t) in Figure 1 ); this is one contribution of the work presented here.. Extensive analysis of Internet data has shown that Internet traffic is LRD (see [1] ). It has been argued that the LRD behaviour of Internet traffic is related to heavy tailed file transfer volumes ( [2] ). Recent studies (see [12] , [14] , [15] ) show that the stationary distribution of a queue fed with LRD traffic will have a non-exponential tail; for example, it has been shown that an arrival rate process auto-covariance that is O( 1 τ α−1 ), 1 < α < 2, leads to a stationary distribution of buffer occupancy that has a tail that is O( 1 x α−1 ). The above observations are usually combined to conclude that the link buffer occupancies in the Internet will be heavy tailed. Such observations are, however, based on an "open loop" analysis of an LRD traffic source feeding a buffer. Recent numerical studies ( [16] , [17] , [4] ) suggest that an understanding of traffic and buffer processes in the Internet should take into account the closed loop nature of Internet congestion control, namely TCP which is an adaptive window protocol (AWP). The second contribution of this paper is to carry out such an analysis for the network scenario of Figure 1 and for a general AWP.. It is easy to see that the behaviour of a buffer for a given input process can be strikingly different in a feedback loop as compared to when the same process is applied to the buffer (i.e., "open loop"). In Figure 2 we provide a simple example. Part (a) of the figure depicts a closed queueing system where a single customer is fed back to the queue (with a new service requirement distributed as exponential(µ)) as soon as it gets served; the system is clearly stable as there is always a single customer in the system. Note that the customer arrival instants to the queue form a Poisson process of rate µ. Part (b) of the figure depicts an M/M/1 queue with a Poisson arrival process of rate µ, and exponentially distributed service requirements with mean 1 µ ; this queue is clearly unstable (the queue length process being a null recurrent Markov chain). It is intuitive that introduction of window reductions due to a packet marking scheme at the link buffer would result in a better behaved buffer occupancy distribution. We study this phenomenon for two specific AWPs and find that the results are in accordance with the intuition. We have developed a framework for the analysis of processes related to the bottleneck link buffer under an adaptive window protocol (AWP) controlled transfer of randomly arriving finite volume files. The most important example of an AWP is TCP. The key idea used in the analysis was to look at the window size of the AWP controlling the transfer of a file as a function of the amount of data served from the file.. We have analysed the auto-covariance function of the process corresponding to the aggregate traffic into the bottleneck link buffer. Bounds on the asymptotic behaviour of the auto-covariance function is given which have explicit dependence on the AWP used and the file transfer volume distribution.. Also analysed is the stationary behaviour of the bottleneck link buffer occupancy under the same scenario. An explicit expression for the stationary distribution of the link buffer occupancy was obtained, and was again seen to have a dependence on the AWP and the file size distribution.. It was shown that, for Pareto distributed file transfer volumes with shape parameter α, 1) When TCP's congestion avoidance or slow start algorithms are used, the traffic into the link buffer is long range dependent (LRD) for small file transfer request arrival rates and for 1 < α < 2, 2) Under the congestion avoidance phase of TCP, the tail of the distribution of the link buffer occupancy process is regularly varying with index 2(α −1). This also implies that the mean link buffer occupancy is finite iff α > 1.5. 3) Under the slow start phase of TCP, the tail of the distribution of the link buffer occupancy process is regularly varying with index (α − 1). This means that the mean link buffer occupancy is finite iff α > 2. Thus we have given an example (Pareto distributed file sizes with 1.5 < α < 2 transferred under the congestion avoidance phase of TCP) where, in the presence of LRD input to the link buffer, the buffer occupancy has finite mean. This suggests that the impact of long range dependence of Internet traffic may not be as severe as is usually predicted by means of an open loop analysis.. We have also considered the effect of window reductions (owing to random packet markings) on the link buffer occupancy process. It was seen that, for an AWP which increases its window aggressively like the TCP does in its slow start phase followed be a response to packet marking which is either gentle as in TCP Tahoe or aggressive as in TCP Reno, the mean link buffer occupancy remains finite irrespective of the file size distribution while the traffic arrival process into the link buffer could still be long range dependent.. The characterisation of the tail of link buffer occupancy we have developed could lead to an explanation of the sensitivity with distribution of TCP throughput performance with finite volume transfers (as observed in [9] ).. The work reported in this paper is for a zero propagation delay link; it will be interesting to study how the results presented in this paper change as the propagation delay increases. APPENDIX I PROOF OF THEOREM IV.1 Let V i (t) and Y i (t), 1 ≤ i ≤ N (t), denote, respectively, the total and residual service requirements of the i th session active at time t. Now,. is the age of the file active at time 0, and U (V 1 (0)) − (V 1 (0) − Y 1 (0)) > τ implies that the source will still be sending data for this file until time τ . Define the last term to be J 3 (τ ).. Plugging the distributions of N (0), V 1 (0) and Y 1 (0) using results given in Section II-A, we get from Equation 15,. At τ there could be other sessions active; these arrive in the interval (0, τ ]. Let us continue to use the index 1 at time τ for the session that was active at time 0. Since Y 1 (τ ) > 0, (and noting that. where N (τ ) is the number of sessions active at τ other than the tagged session which was active at time 0. The inequality is obtained since there could be a positive rate from the other sessions at τ . By hypothesis, we have 0 < r ≤ inf The proof follows with a change of variable (using y = v − u). where v(n) is a row vector of dimension n with components v 1 , ..., v n and y(n) is a row vector of dimension n with components y 1 , ..., y n . Also, abusing notation, is a function only of τ − θ.

B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O