PAC Analogues of Perceptron and Winnow Via Boosting the Margin We describe a novel family of PAC model algorithms for learning linear threshold functions. The new algorithms work by boosting a simple weak learner and exhibit sample complexity bounds remarkably similar to those of known online algorithms such as Perceptron and Winnow, thus suggesting that these well-studied online algorithms in some sense correspond to instances of boosting. We show that the new algorithms can be viewed as natural PAC analogues of the online p-norm algorithms which have recently been studied by Grove). As special cases of the algorithm, by taking p = 2 and p = ∞ we obtain natural boosting-based PAC analogues of Perceptron and Winnow respectively. The p = ∞ case of our algorithm can also be viewed as a generalization (with an improved sample complexity bound) of Jackson and Craven's PACmodel boosting-based algorithm for learning "sparse perceptrons" (Jackson & Craven, 1996 , Advances in neural information processing systems 8, MIT Press). The analysis of the generalization error of the new algorithms relies on techniques from the theory of large margin classification. One of the most fundamental problems in computational learning theory is that of learning an unknown linear threshold function from labeled examples. Many different learning algorithms for this problem have been considered over the past several decades. In particular, in recent years many researchers have studied simple online additive and multiplicative update algorithms, namely the Perceptron and Winnow algorithms and variants thereof (Auer & Warmuth, 1995; Baum, 1990; Bylander, 1998; Freund & Schapire, 1999; Gentile & Littlestone, 1999; Grove, Littlestone, & Schuurmans, 1997; Kivinen, Warmuth, & Auer, 1997; Littlestone, 1988 Littlestone, , 1991 Servedio, 1999; Schmitt, 1998) .. This paper takes a different approach. We describe a natural parameterized family of boosting-based PAC algorithms for learning linear threshold functions. The weak hypotheses used are linear functionals and the strong classifier obtained is a linear threshold function. Although our new algorithms are conceptually and algorithmically very different from Perceptron and Winnow, we establish performance bounds for the new algorithms which are remarkably similar to those of Perceptron and Winnow; we thus refer to the new algorithms as PAC analogues of Perceptron and Winnow. We hope that the analysis of these new algorithms will yield fresh insights into the relationship between boosting and online algorithms.. We give a unified analysis of our Perceptron and Winnow analogues which includes many other algorithms as well. Grove, Littlestone, and Schuurmans (1997) have shown that Perceptron and (a version of) Winnow can be viewed as the p = 2 and p → ∞ cases of a general online p-norm linear threshold learning algorithm, where p ≥ 2 is any real number. We present PAC-model boosting-based analogues of these online p-norm algorithms for any value 2 ≤ p ≤ ∞. The PAC-model Perceptron and Winnow analogues mentioned above are respectively the p = 2 and p = ∞ cases of this general algorithm.. The p = ∞ case of our algorithm can also be viewed as a generalization of Jackson and Craven's PAC-model algorithm for learning "sparse perceptrons" (Jackson & Craven, 1996) . Their algorithm boosts using weak hypotheses which are single Boolean literals; this is similar to what the p = ∞ case of our algorithm does. Our analysis of the p = ∞ case generalizes their algorithm to deal with real-valued rather than Boolean input variables, thus achieving a goal stated in Jackson and Craven (1996) , and also yields a substantially stronger sample complexity bound than was established in Jackson and Craven (1996) .. Section 2 of this paper contains preliminary material, including an overview of the online p-norm algorithms from Gentile and Littlestone (1999) and Grove, Littlestone, and Schuurmans (1997) . In Section 3 we present a simple PAC-model p-norm algorithm and prove that it is a weak learning algorithm for all 2 ≤ p < ∞. In Section 4 we apply techniques from the theory of large margin classification to show how our weak learning algorithm can be boosted to a strong learning algorithm with small sample complexity. Finally, in Section 5 we compare our PAC algorithms with the analogous online algorithms, extend our algorithm to the case p = ∞, and discuss the relationship between the p = ∞ case of our algorithm and the Jackson-Craven algorithm for learning sparse perceptrons. The sample complexity of our boosting-based p-norm PAC learning algorithm is remarkably similar to that of the PAC-transformed online p-norm algorithms of Section 2.1. Both sets of bounds are essentially linear in −1 and quadratic in ũ q X p /δũ ,X . Comparing the bounds in more detail, we see that our sample complexity bound contains various log factors which are not present in the bound for the online variant described in part (a) of Theorem 1. These log factors stem from the bounds given in Theorem 7 and Lemma 12; we do not know if they are inherent in the algorithm's performance or an artifact of the analysis. On the other hand, the bound of variant (a) has an extra factor of p − 1 which is not present in the sample complexity of our algorithm. Results of Gentile and Littlestone (1999) suggest that the most meaningful range for p is 2 ≤ p ≤ O(log n), in which case this factor is quite small. We note that when p = (log n) Gentile and Littlestone have given alternative expressions for the online p-norm bounds in terms of X ∞ and ũ 1 . For these values of p, using an entirely similar analysis the bounds of our algorithm can be analogously rephrased in terms of X ∞ and ũ 1 as well. Several authors have previously studied linear threshold learning algorithms which work by combining weak predictors. Freund and Schapire have studied an algorithm which predicts using a weighted vote of the hypotheses which the Perceptron algorithm generates during its training phase (Freund & Schapire, 1999) . The weight of each hypothesis in this vote is proportional to its survival time, i.e. the number of examples which elapse before it classifies an example incorrectly and causes the Perceptron algorithm to generate a new hypothesis. Freund and Schapire prove generalization error bounds on the resulting classifier which are similar to Vapnik's generalization error bounds for the "maximal margin" hyperplane (Vapnik, 1998) . The Freund-Schapire algorithm differs from our approach in several ways: for one thing, their algorithm is unlike ours in that it does not use boosting to combine the weak predictors. Additionally, whereas our algorithm's final hypothesis is a single linear threshold function, their algorithm's final hypothesis is a depth-2 threshold circuit (a weighted vote over Perceptron hypotheses which are themselves linear threshold functions).. Ji and Ma have suggested that a random-search-and-test approach can be used to find weak classifier linear threshold functions for certain restricted halfspace learning problems (Ji & Ma, 1997) . They propose combining these weak classifier linear threshold functions with a simple majority vote; thus, their approach also results in a final hypothesis which is a depth 2 threshold circuit.. Our approach is closest to that of Jackson and Craven, who use boosting to combine single literals into a strong hypothesis linear threshold function. As we show in Section 5, the p = ∞ case of our algorithm strengthens and generalizes their results.. The close similarity in performance bounds between our boosting-based algorithms and the online p-norm algorithms suggests a relationship between boosting and online learning. Freund and Schapire (1996) and Schapire (1999) have investigated this relationship in the context of game theory.

O O O O O O O B O O O O O O O O O O O O O B I I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O


The effect of packet dispersion on voice applications in IP networks Abstract-Delivery of real time streaming applications, such as voice and video over IP, in packet switched networks is based on dividing the stream into packets and shipping each of the packets on an individual basis to the destination through the network. The basic implicit assumption on these applications is that shipping all the packets of an application is done, most of the time, over a single path along the network. In this work, we present a model in which packets of a certain session are dispersed over multiple paths, in contrast to the traditional approach. The dispersion may be performed by network nodes for various reasons such as load-balancing, or implemented as a mechanism to improve quality, as will be presented in this work. To study the effect of packet dispersion on the quality of voice over IP (VoIP) applications, we focus on the effect of the network loss on the applications, where we propose to use the Noticeable Loss Rate (NLR) as a measure (negatively) correlated with the voice quality. We analyze the NLR for various packet dispersion strategies over paths experiencing memoryless (Bernoulli) or bursty (Gilbert model) losses, and compare them to each other. Our analysis reveals that in many situations the use of packet dispersion reduces the NLR and thus improves session quality. The results suggest that the use of packet dispersion can be quite beneficial for these applications.Index Terms-Bursty losses, noticeable loss rate, packet dispersion, stochastic processes, voice over IP quality.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O B I I B I B I B I I I


The Promise and Perils of Near-Regular Texture Motivated by the low structural fidelity for near-regular textures in current texture synthesis algorithms, we propose and implement an alternative texture synthesis method for near-regular texture. We view such textures as statistical departures from regular patterns and argue that a thorough understanding of their structures in terms of their translation symmetries can enhance existing methods of texture synthesis. We demonstrate the perils of texture synthesis for near-regular texture and the promise of faithfully preserving the regularity as well as the randomness in a near-regular texture sample. One obvious limitation of this work is its focus on near-regular texture alone. Nevertheless, several fundamental issues for texture understanding and synthesis seem to be related. Firstly, with respect to the very different properties of different textures (random versus regular) should we treat all textures uniformly? If not, how should we combine different methodologies together? Secondly, almost all the texture synthesis al-gorithms have to define a window, sometime called patch, and which we call a tile, to sample the original input textures. (In this paper, we use the word window, patch and tile interchangeably). What are the basic variables involved in choosing a window, and what are their impacts? Thirdly, for near-regular textures, can we do better than what has been proposed here? What will happen if we go beyond translation symmetry and investigate the effect of rotation, reflection and glide-reflection symmetries? and how may this be related to the concept of texton for near-regular texture?. In the following, under the context of texture synthesis, we shall elaborate on each of these topics in more detail. Near-regular textures are common in our daily life. They can be observed in man-made products, by hand or by machine, ranging from buildings to fabrics, as well as in nature and biological process of life science (Feynman, 1998; Chambers, 1995; Senechal, 1995; Zee, 1999; Hargittai and Hargittai, 2000) . Humans have an innate ability to perceive and take advantage of symmetry (Leyton, 1992) . Rao and Lohse (1993) showed that regularity plays an important role in human texture perception. However, it is not obvious how to automate this powerful insight.. Mathematically speaking, regular texture refers to periodic patterns that present non-trivial translation symmetry, with the possible addition of rotation, reflection and glide-reflection symmetries (Miller Jr., 1972; Coxeter, 1980; Grünbaum and Shephard, 1987) . When studying periodic patterns, a useful fact from mathematics is the answer to Hilbert's 18th prob- * This work is funded in part by an NSF research grant # IIS-0099597. lem: there is only a finite number of symmetry groups for all possible periodic patterns in dimension n (Bieberbach, 1910) . When n = 1 there are seven frieze groups, and when n = 2 there are 17 wallpaper groups. Here group is referring to the symmetry group of a periodic pattern. A symmetry group is composed of transformations that keep the pattern setwise invariant.. In computer vision and computer graphics, the application of this classic mathematics for regular or nearregular pattern analysis has yet to be fully explored. Only recently, have computer algorithms of symmetry group classification been developed for periodic patterns in real images under Euclidean (Liu and Collins, 2000; and affine transformations (Liu and Collins, 2001) , based on a careful analysis of the basic tile shapes of regular patterns. In computer graphics, one interesting recent work (Kaplan and Salesin, 2000) is to find Escher-like tilings by deforming a single closed planar figure to tile a plane.. Near-regular texture is referring to textures that are not strictly symmetrical. The irregularity can be caused by various statistical departures from regular textures. These departures can happen along different dimensions of symmetry (Liu, 2001) , for example, color (single, multi) (Tsin et al., 2001) , intensity (irregular statistical alterations, random noise), global or local geometric deformations (affine, projective, random) (Liu and Collins, 2000, 2001) , and resolution. See Fig. 1 from Liu (2001) for some examples of symmetry dimensions. The focus of this paper is on faithful texture synthesis of near-regular textures where departure from regularity is primarily caused by statistical color and intensity variations, while the underlying structural regularity remains. There are many examples of this type of near-regular textures, e.g. brick walls, tiled floors, carpets, and woven sheets, where the texture patterns (each brick, tile, straw or bamboo strip) vary only locally. The idea of viewing a random texture as a distorted version of a regular texture was expressed in an early paper by Zucker (1976) . More recently, we have demonstrated a computational model for near-regular textures that vary along geometry, lighting and color dimensions .. Existing work on texture synthesis has achieved impressive results for a variety of different types of textures (e.g., De Bonet, 1997; Efros and Leung, 1999; Ashikhmin, 2001; Efros and Freeman, 2001; Hsu and Wilson, 1998; Wei and Levoy, 2000; Hertzmann et al., 2001; Xu et al., 2001; Liang et al., 2001; Zhu et al., 2000; Kwatra et al., 2003; Cohen et al., 2003) . These texture synthesis algorithms share a common theme of local neighborhood-based statistical approaches. Distinctions can be drawn between approaches that constructively establish statistical models for the input texture (Cross and Jain, 1983; Zhu et al., 1997) versus others that seek to find matching joint statistics directly in the input samples (De Bonet, 1997; Portilla and Simoncelli, 2000; Zhu et al., 2000) . More recently, non-parametric estimation of texture PDFs has become popular (Efros and Leung, 1999; Wei and Levoy, 2000; Efros and Freeman, 2001; Liang et al., 2001) . These texture synthesis algorithms are relatively simple to implement, fast to run (Wei and Levoy, 2000; Liang et al., 2001) and able to reproduce a large variety of textures, from regular to random, as claimed by the authors. However, after reviewing the results of existing work applied to near-regular textures, we observe that the structural regularity is usually not well preserved in the synthesized texture. This is especially true when the input sample has interlocking near-regular patterns, or is oriented obliquely. For example, we have not yet seen an existing texture synthesis algorithm that preserves the regularity in a brick wall sample ( Fig. 2(a) ). In addition, the structural property of near-regular textures has not been used as an objective measure for texture synthesis algorithms .. This situation motivates us to propose and implement an alternative texture synthesis method for near-regular texture that is particularly faithful to its structural property while preserving the randomness observed in the input data. Figures 2 and 3 demonstrate two sample results from our texture synthesis algorithm in contrast to the texture synthesis results reported in Efros and Freeman (2001) .. Section 2 defines basic properties of regular texture such as generating tile, symmetry groups and lattice types. In Section 3 we explain our texture analysis and synthesis algorithm and demonstrate some experimental results. Section 4 discusses several relevant issues in near-regular texture synthesis, from window size to the concept of textons. Section 5 concludes with a summary and future research directions.

O O O O O B I O O O O O O O O O O O B I O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O


Skew constacyclic codes over Galois rings We generalize the construction of linear codes via skew polynomial rings by using Galois rings instead of finite fields as coefficients. The resulting non commutative rings are no longer left and right Euclidean. Codes that are principal ideals in quotient rings of skew polynomial rings by a two sided ideals are studied. As an application, skew constacyclic self-dual codes over GR(4 2 ) are constructed. Euclidean self-dual codes give self-dual Z 4 −codes. Hermitian self-dual codes yield 3−modular lattices and quasi-cyclic self-dual Z 4 −codes. Polynomial rings and their ideals are essential to the construction and understanding of cyclic codes. For the first time in [5] non commutative skew polynomial rings have been instead of linearized polynomials used to construct a generalization of cyclic codes. In that approach, it is necessary to use as alphabet a finite field with a non trivial Galois automorphism, like, e.g. F 4 . In the present work we extend that approach by considering as alphabets Galois rings of even characteristic like, e.g. GR(4 2 ). The technical difficulty in passing from field alphabet to ring alphabet is that the skew polynomial rings are not Ore rings; in particular they are no longer left and right Euclidean. However, left and right division by unitary polynomials are still well defined. Therefore codes that are principal ideals generated by unitary polynomials in quotient rings of skew polynomial rings by a two sided ideal are studied. In particular the problem of finding central polynomials, that is generators of principal two sided ideals is addressed.. As an application self-dual codes over GR(4 2 ) are constructed, and used for three of the four applications of [14] . 1)Self-dual Euclidean codes give self-dual Z 4 codes by projection on a trace orthogonal basis. Many Type I codes in length 24 are obtained and classified by the root systems of their Construction A lattice. New coding constructions of the Odd Leech lattice, the only unimodular lattice of norm 3 in dimension 24 are given thus supplementing the results of [15] . Some new Type II codes in length 24 are also obtained, that yield the Leech lattice by Construction A, though not being Lee-optimal in the sense of Rains [20] . A Type I code in length 40 is constructed that is not Type IV. In fact its Lee and Euclidean distance are better than what would be possible for a Type IV code of that length [9] .. 2)Self-dual Hermitian codes build 3-modular lattices. In particular a simpler construction of one of the two extremal such lattices in dimension 28 is given.. 3)Self-dual Hermitian codes yield self-dual quasi-cyclic codes over Z 4 by the cubic construction. Self-dual Type II codes are obtained in length 24 and classified by their symmetrized weight enumerator and root system.. The material is organized as follows. Section 1 contains generalities on skew polynomial rings over Galois rings. Section 2 defines the codes generated by principal ideals in quotient rings where X n − 1 is replaced by a central polynomial. Section 3 explains how to generate central polynomials by using the notion of bound and gives a few examples. Section 4 considers parity check matrices and duals of the constacyclic codes defined before. Section 5 constructs Euclidean self-dual codes over GR(4 2 ). Section 6 is devoted to self-dual Z 4 codes and their unimodular lattices. Section 7 studies Hermitian self-dual codes over GR(4 2 ) and 3−modular lattices. Section 8 considers the cubic construction of self-dual Z 4 codes from Hermitian self-dual codes over GR(4 2 ).

O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O B I O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O


Numerical simulation of clarifier-thickener units treating ideal suspensions with a flux density function having two inflection points Abstract. We consider a nonconvex conservation law modelling the settling of particles in ideal clarifier-thickener units. The flux function of this conservation law has an explicit spatial dependence that is discontinuous. Previous works by two of the authors, together with collaborators, have been aimed at providing a firm rigorous ground of mathematical (global existence and uniqueness) and numerical analysis for clarifier-thickener models. Although the results of these works are briefly summarized herein, the chief goal of this paper is to present a number of numerical simulations of practical interest and to draw some conclusions from them. In contrast to previous papers we consider here flux density functions with two inflection points, which results in solutions exhibiting a richer structure than for flux density functions having one inflection point. The relevance of the "two inflection point" case comes from experimental observations. In addition, we include here several numerical simulations in which the feed rate and overflow and underflow bulk rates vary with time. Time dependent situations have high practical value, but have been very little studied in the literature. The main assumption of the one-dimensional kinematic sedimentation theory, which originated in Kynch's paper [1] , states that in an ideal suspension of small equal-sized solid particles dispersed in a viscous fluid, considered as a superposition of two continuous phases, the solid-fluid relative or slip velocity v r is a function of the local solids concentration u only. (In this paper, we assume that u is the dimensionless solids volume fraction.) Defining the Kynch batch flux density function h(u) := u(1 − u)v r (u), we can then state the governing equation for batch settling of a suspension in a cylindrical closed vessel as the scalar conservation law ∂u ∂t + ∂h(u) ∂x = 0, (1.1) where t is time and x is depth. The function h(u) depends on the material specific properties of the mixture under study; precise assumptions on h(u) will be stated later. Numerous studies (see [2, 3] for overviews) and extensive use in industry have established the kinematic sedimentation theory as a useful description for the settling of a variety of materials including mineral tailings, wastewater and blood. Practitioners very soon used this theory for design calculations of continuously operated settling tanks, so-called thickeners, by adding a transport term q(t)u to the flux density function h(u), where q(t) is a controllable mixture flow velocity [4, 5] . This results in the equation. Several researchers recognized that information on whether a cylindrical continuous thickener is able to treat a given suspension with a known function h(u) under stationary flow conditions can essentially be read off from plots of the continuous flux function qu + h(u) (with q = q(t) kept constant) [6, 7, 8, 9] . However, to use (1.2) for the simulation of continuous sedimentation, one needs to explicitly model the feed and discharge mechanisms and to provide (in the widest sense) boundary conditions. Petty [10] was the first to propose boundary conditions given as explicit boundary concentration values (Dirichlet boundary conditions) for Eq. (1.2), but recognized that the resulting model was not well-posed since overflow and underflow concentration waves may break through the feed and discharge boundary levels, such that in dependence of the solution u, prescribed boundary data are ignored. The well-posedness of a scalar conservation law with boundary conditions was recovered by the concept of entropy boundary conditions [11, 12, 13] . In a series of papers, Bustos and her co-workers utilized this new concept to provide a rigorous mathematical framework to continuous sedimentation [14, 15, 16] .. In spite of its amenability to mathematical analysis, the Petty-Bustos model [10, 14, 15, 16] suffers from some drawbacks. Among them is the lack of a global conservation principle due to the use of Dirichlet boundary conditions, and the complete neglect of a clarification zone. It is preferable to replace the Dirichlet boundary conditions by transitions between the transport flux q(x, t)u and the composite flux q(x, t)u + h(u), which leads to a pure initial value problem. Moreover, the feed suspension should enter between the overflow outlet at the top and the discharge outlet at the bottom. This gives rise to an upwards-directed volume average velocity q L ≤ 0 above and a downwards-directed velocity q R ≥ 0 below the feed level. The feed source itself is modeled by a singular source term.. Consider now a continuously operated axisymmetric clarifier-thickener vessel as drawn in three variants in Figure 1 . Combining all these ingredients, we obtain the final clarifier-thickener model as a conservation law with a flux function that varies discontinuously across the discharge level x = x R , the feed level x = 0 and the overflow level x = x L . The discontinuous flux precludes applying standard conservation law existence and uniqueness theory, and has turned the clarifierthickener model into a topic of fairly recent mathematical research. Thorough constructions and classifications of solutions of clarifier-thickener models were presented in a series of papers by Diehl, see [17, 18, 19, 20, 21] . Two of the authors, together with collaborators, have in a series of papers [22, 23, 24, 25, 26, 27, 28] provided a firm rigorous ground of mathematical and numerical analysis for (first and second order) clarifier-thickener models, where a suitable notion of solution has been introduced, uniqueness of such a solution has been established, and existence of such a solution has been proved by showing that certain numerical schemes converge. The numerical schemes can be also employed as simulation tools for clarifier-thickener models.. To put the present paper in the perspective of our previous treatments, let us mention that the starting point of our analysis was Gimse and Risebro's analysis [29] of the Riemann problem for a conservation law with two different fluxes and two different constant initial data adjacent to x = 0. The solution of the Riemann problem forms the building block of the very efficient front tracking method [30] , which consists in determining an approximate solution to a conservation law with piecewise constant initial data by solving Riemann problems and tracking the fronts (and their collisions) that form their solution. For a conservation law with discontinuous flux, the general front tracking procedure, along with a proof of its well-posedness, was given in [31] for a particular model of two-phase flow in porous media, see also [32, 33] . For another approach to the two-phase flow model, see [34] . The approach in [31, 32, 33] was adapted to the clarifier-thickener model in [23] , and it was shown that this method approximates a weak solution of the clarifier-thickener problem. On the other hand, it requires some effort in implementation, so at the same time we started to analyze easier to implement finite difference methods for the clarifier-thickener model. In this sense, the simplest finite difference scheme for the clarifier-thickener model is based on relaxation, and has been shown to converge to weak solution via the compensated compactness method [24, 35] . However, more accurate numerical schemes, including the scheme used in this paper, are based on the Engquist-Osher [36] generalized upwind finite difference method. This method was applied to the clarifier-thickener model for the first time in the 'engineering' paper [ 37] , but see also [38, 39] . It is straightforward to extend this method to vessels with varying cross-sectional area, see [26] for numerical examples and [25] for a proof of the convergence of this extension to a weak solution. For further details on the finite difference scheme we refer to Section 3.. It is well known that weak (discontinuous) solutions to conservation laws are not unique, and that a selection criterion or entropy condition is necessary to single out a unique physically relevant solution, the entropy solution. For the clarifier-thickener model, an entropy solution concept is presented in [27] , which in turn was based on the works [38, 39, 40] . This concept extends the classical approach due to Kružkov [41] to conservation laws with discontinuous flux. In particular, it characterizes admissible concentration jumps across the discontinuity levels x = x R , x = 0 and x = x L . In fact, it turns out that within the entropy concept outlined in [27] , these jumps are minimal in the sense of the 'minimal jump condition' postulated in [29, 31] . In [27] we prove that an entropy weak solution of the clarifier-thickener model is unique, and show that the finite difference scheme previously advanced in [37] converges to this solution, see also [38, 39, 40, 42, 43] . Thus, the convergence proof acts as a means to show existence of an entropy weak solution. The analysis is valid for a very general class of flux functions, including the ones studied in this paper, and is summarized in Section 3.. In some cases, real-world suspensions cannot be characterized as an ideal suspension. We mention that the clarifier-thickener model has been extended to polydisperse suspensions with particles differing in size or density [22] , and to flocculated suspensions forming compressible sediments [28, 44] . In the latter case, our model is extended by a strongly degenerating diffusion term modelling sediment compressibility. Drawing from the analysis in [40, 42] , we prove in [28] uniqueness of entropy solutions and show that the scheme studied herein, appropriately extended to handle the diffusion term, converges to a weak solution that satisfies an entropy condition.. In our previous papers [22, 23, 24, 25, 26, 27, 28] , we focused on the exposition of the theoretical background of the clarifier-thickener model and its numerical analysis and presented only a few numerical examples. The choice of examples in the engineering papers [37, 44] is also very limited. Meanwhile, a large number of simulations of cases of practical interest have been performed by the last author in his engineering thesis [45] . It is the main purpose of this paper to present further examples of transient behaviour predicted by the clarifier-thickener model. In contrast to all above-cited previous papers, we include here the flux density function with two inflection points proposed by Barton et al. [46] (see also [47] ). The general mathematical framework is, of course, independent of this number of inflection points, so that the well-posedness results and the numerical scheme clearly handle all possible functions h(u) with two inflection points. However, numerical solutions for functions with two or more inflection points in general exhibit a richer structure than for flux density functions having one inflection point only (the latter class includes the famous Richardson-Zaki [48] and Vesilind [49] functions). The relevance of the former case accrues from experimental observations [50, 51, 52, 53, 54, 55] indicating that flux functions with two inflection points describe the material behaviour of many materials more accurately than, say, a simple Richardson-Zaki type [48] flux. In many cases (but with the exception of [53] ), the apparent diminishment of the flux density function at higher solids concentrations, which causes the appearance of more than one inflection point, is attributed to solids stress effects. This phenomenon has been explicitly incorporated in [28, 44] in an alternative way by a degenerate diffusion term, but the present approach is algebraically simpler. (The most accurate way to consider solids stress probably consists in a combination of both methodologies, see e.g. [56] .) We do not regard two or more inflection points as a mandatory ingredient of the model, and they may even be mechanistically inappropriate for some materials, but derive our interest in this case from the above-cited experimental literature. Sedimentation experiments are usually conducted for batch settling of a suspension in a closed column, so the present work can be understood as an extrapolation of experimental findings to continuously operated units. Unfortunately, experimental information on concentration measurements for the latter case is very scarce, but we refer to the papers [53, 57, 58, 59, 60, 61] for explicit comparisons of batch settling experiments with theoretical solutions of the Kynch model, to which our model reduces if bulk velocities are set to zero.. On the other hand, in the numerical experiments of our previous papers the only control variable whose value is changed in time is the solids feed concentration. However, in realistic operation also the feed rate and overflow and underflow bulk rates should vary, as for example in a wastewater treatment plant whose load will usually depend on daytime, and therefore obeys a 24-hour periodic pattern. Such a case was considered by Attir and Denn in [62] and simulated by an ad-hoc numerical scheme introduced in [63] . We have included here one example in which the volume flows are changed in a stepwise fashion, and another one showing the response of the unit to periodic loading.. The remainder of the paper is organized as follows. In Section 2 the governing equation for continuous sedimentation is derived in some detail. In the formulation of the final initial value problem we add the feed source to the model for constant cross-sectional area formulated in [23, 27, 37] . Moreover, weak solutions are defined and the main convergence result is stated. In Section 3 the numerical method is outlined and its basic properties are stated. In Section 4 we present numerical examples in the application to clarifier-thickener models. A discussion of the numerical examples and a short conclusion of the paper are provided in Section 5. We briefly comment the numerical results. Figures 5 and 7 illustrate that solutions produced with a flux density function h(u) having two inflection points, whose location will not change if we add the linear flux q L (u − u F ) or q R (u − u F ), in general show a more complicated solution structure than analogous cases produced by the same parameters q L , q R and q F but using a function h(u) having one inflection point only. Figure 8 alerts to the way how the evolution of concentration profiles is influenced by the vessel geometry. We observe that the conical segment of the thickening zone of Vessel 2 leads to a continuously varying concentration profile. Moreover, since the settleable area of Vessel 2 is smaller than that of the cylindrical Vessel 1, the unit is filled up, and the overflow of dilute suspension is attained more rapidly in Vessel 2 than in Vessel 1.. In Figure 9 (a) we observe that under these new operating conditions, the clarifier-thickener unit is filled up: the thickening zone is filled up with sediment, which breaks through the feed level x = 0 near t = 6 × 10 5 s. The suspension-supernate level continues to rise until, finally, solid material breaks through the overflow level. By t = 1.2 × 10 6 s the system has attained a steady state at which the feed suspension of concentration u F = 0.15 is simultaneously concentrated to a sediment of concentration 0.9 and, by the clarification zone, dilute to an overflow effluent of a concentration less than 0.1. Figure 9 new steady state is attained, in which only the thickening zone handles the feed flux. The final steady-state concentration of the sediment leaving the clarifier-thickener, u R , is given here through the global balance (Q R − Q L )u F = Q R u R , which implies. Finally, in Figure 10 the sediment level oscillates around the fed level x = 0 in an apparently periodic pattern, while the concentration profile in inferior parts of the thickening zone becomes stationary after several days.. In response to concerns risen by one reviewer of the present paper, let us emphasize that it is well possible, and no violation of conservation of mass, that within the clarification zone x L < x < 0, the local solids concentration exceeds the feed concentration. This occurs, of course, in transient situations where, for example, the clarifier-thickener is operated at steady state with u F > 0, the clarification zone contains solids of positive concentration, and at some time we immediately turn off the feed mechanism. Then a short time interval elapses until the clarifier-thickener empties. Clearly, during that time interval the above mentioned situation occurs. However, in Figures 7 (a) , (d) and 9 (b), we observe that the steady state (non-transient) concentration in the clarification zone exceeds the feed concentration. To see that this observation does not place in doubt the model's verisimilitude, let us restrict ourselves to a cylindrical vessel and assume that u L and u R are the steady state clarified liquid and sediment concentrations, that is, the constant values the steady-state solution assumes for x < x L and x > x R , respectively. Then a global mass balance yields. which can be rearranged to give. and thus. which implies that except in the cases α ∈ {0, 1}, a steady state with u R > u F leads to an effluent concentration u L < u F . However, if we denote by u clarif the steady-state concentration in the clarification zone, then the Rankine-Hugoniot condition (3.14) across x L is Figure 9 . Simulation of the dynamics of a clarifier-thickener with varying crosssectional area under variable, piecewise constant feed conditions. which implies Figure 10 . Simulation of the dynamics of a clarifier-thickener with varying crosssectional area under continuously varying, periodic feed conditions. geometry). On the other hand, it has been demonstrated that a flux density function with two inflection points can lead to fairly complex solution patterns (compared with a function having one inflection point only) even when all control variables are kept constant. Finally, it should be emphasized that the whole model is backed up by a well-posedness (existence and uniqueness) theory.

B I O B O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Random Spanning Trees and the Prediction of Weighted Graphs We investigate the problem of sequentially predicting the binary labels on the nodes of an arbitrary weighted graph. We show that, under a suitable parametrization of the problem, the optimal number of prediction mistakes can be characterized (up to logarithmic factors) by the cutsize of a random spanning tree of the graph. The cutsize is induced by the unknown adversarial labeling of the graph nodes. In deriving our characterization, we obtain a simple randomized algorithm achieving in expectation the optimal mistake bound on any polynomially connected weighted graph. Our algorithm draws a random spanning tree of the original graph and then predicts the nodes of this tree in constant expected amortized time and linear space. Experiments on real-world datasets show that our method compares well to both global (Perceptron) and local (label propagation) methods, while being generally faster in practice. A widespread approach to the solution of classification problems is representing datasets through a weighted graph where nodes are the data items and edge weights quantify the similarity between pairs of data items. This technique for coding input data has been applied to several domains, including Web spam detection [19] , classification of genomic data [27] , face recognition [10] , and text categorization [13] . In many applications, edge weights are computed through a complex datamodelling process and typically convey information that is relevant to the task of classifying the nodes.. In the sequential version of this problem, nodes are presented in an arbitrary (possibly adversarial) order, and the learner must predict the binary label of each node before observing its true value. Since real-world applications typically involve large datasets (i.e., large graphs), online learning methods play an important role because of their good scaling properties. An interesting special case of the online problem is the so-called transductive setting, where the entire graph structure (including edge weights) is known in advance. The transductive setting is interesting in that the learner has the chance of reconfiguring the graph before learning starts, so as to make the problem look easier. This data preprocessing can be viewed as a kind of regularization in the context of graph prediction.. When the graph is unweighted (i.e., when all edges have the same common weight), it was found in previous works [17, 16, 14, 15, 8] that a key parameter to control the number of online prediction mistakes is the size of the cut induced by the unknown adversarial labeling of the nodes, i.e., the number of edges in the graph whose endpoints are assigned disagreeing labels. However, while the number of mistakes is obviously bounded by the number of nodes, the cutsize scales with the number of edges. This naturally led to the idea of solving the prediction problem on a spanning tree of the graph [7, 18, 19] , whose number of edges is exactly equal to the number of nodes minus one. Now, since the cutsize of the spanning tree is smaller than that of the original graph, the number of mistakes in predicting the nodes is more tightly controlled. In light of the previous discussion, we can also view the spanning tree as a "maximally regularized" version of the original graph.. Since a graph has up to exponentially many spanning trees, which one should be used to maximize the predictive performance? This question can be answered by recalling the adversarial nature of the online setting, where the presentation of nodes and the assignment of labels to them are both arbitrary. This suggests to pick a tree at random among all spanning trees of the graph so as to prevent the adversary from concentrating the cutsize on the chosen tree [7] . Kirchoff's equivalence between the effective resistance of an edge and its probability of being included in a random spanning tree allows to express the expected cutsize of a random spanning tree in a simple form. Namely, as the sum of resistances over all edges in the cut of G induced by the adversarial label assignment.. Although the results of [7] yield a mistake bound for arbitrary unweighted graphs in terms of the cutsize of a random spanning tree, no general lower bounds are known for online unweighted graph prediction. The scenario gets even more uncertain in the case of weighted graphs, where the only previous papers we are aware of [16, 14, 15] essentially contain only upper bounds. In this paper we fill this gap, and show that the expected cutsize of a random spanning tree of the graph delivers a convenient parametrization 1 that captures the hardness of the graph learning problem in the general weighted case. Given any weighted graph, we prove that any online prediction algorithm must err on a number of nodes which is at least as big as the expected cutsize of the graph's random spanning tree (which is defined in terms of the graph weights). Moreover, we exhibit a simple randomized algorithm achieving in expectation the optimal mistake bound to within logarithmic factors. This bound applies to any sufficiently connected weighted graph whose weighted cutsize is not an overwhelming fraction of the total weight.. Following the ideas of [7] , our algorithm first extracts a random spanning tree of the original graph. Then, it predicts all nodes of this tree using a generalization of the method proposed by [18] . Our tree prediction procedure is extremely efficient: it only requires constant amortized time per prediction and space linear in the number of nodes. Again, we would like to stress that computational efficiency is a central issue in practical applications where the involved datasets can be very large. In such contexts, learning algorithms whose computation time scales quadratically, or slower, in the number of data points should be considered impractical.. As in [18] , our algorithm first linearizes the tree, and then operates on the resulting line graph via a nearest neighbor rule. We show that, besides running time, this linearization step brings further benefits to the overall prediction process. In particular, similar to [16, Theorem 4.2] , the algorithm turns out to be resilient to perturbations of the labeling, a clearly desirable feature from a practical standpoint.. In order to provide convincing empirical evidence, we also present an experimental evaluation of our method compared to other algorithms recently proposed in the literature on graph prediction. In particular, we test our algorithm against the Perceptron algorithm with Laplacian kernel by [16, 19] , and against a version of the label propagation algorithm by [31] . These two baselines can viewed as representatives of global (Perceptron) and local (label propagation) learning methods on graphs. The experiments have been carried out on five medium-sized real-world datasets. The two tree-based algorithms (ours and the Perceptron algorithm) have been tested using spanning trees generated in various ways, including committees of spanning trees aggregated by majority votes. In a nutshell, our experimental comparison shows that predictors based on our online algorithm compare well to all baselines while being very efficient in most cases.. The paper is organized as follows. Next, we recall preliminaries and introduce our basic notation. Section 2 surveys related work in the literature. In Section 3 we prove the general lower bound relating the mistakes of any prediction algorithm to the expected cutsize of a random spanning tree of the weighted graph. In the subsequent section, we present our prediction algorithm WTA (Weighted Tree Algorithm), along with a detailed mistake bound analysis restricted to weighted trees. This analysis is extended to weighted graphs in Section 5, where we provide an upper bound matching the lower bound up to log factors on any sufficiently connected graph. In Section 6, we quantify the robustness of our algorithm to label perturbation. In Section 7, we provide the constant amortized time implementation of WTA. Based on this implementation, in Section 8 we present the experimental results. Section 9 is devoted to conclusive remarks. With the above notation and preliminaries in hand, we now briefly survey the results in the existing literature which are most closely related to this paper. Further comments are made at the end of Section 5.. Standard online linear learners, such as the Perceptron algorithm, are applied to the general (weighted) graph prediction problem by embedding the n vertices of the graph in R n through a map i → K −1/2 e i , where e i ∈ R n is the i-th vector in the canonical basis of R n , and K is a positive definite n × n matrix. The graph Perceptron algorithm [17, 16] bound is weight-scale independent, but the interplay between the two factors in it may lead to a vacuous result. At a given scale for the weights w i,j , if G is dense, then we may have. G may become as large as n.. The idea of using a spanning tree to reduce the cutsize of G has been investigated by [19] , where the graph Perceptron algorithm is applied to a spanning tree T of G. The resulting mistake bound is of the form Φ [19] suggest to apply the graph Perceptron algorithm to the spanning tree T with smallest geodesic diameter. The geodesic diameter of a weighted graph G is defined by. where the minimum is over all paths Π i,j between i and j. The reason behind this choice of T is that, for the spanning tree T with smallest geodesic diameter, it holds that. , and on the other hand the adversary may still concentrate all φ-edges on the chosen tree T , so there is no guarantee that Φ W T (y) remains small either.. [18] introduce a different technique showing its application to the case of unweighted graphs. After reducing the graph to a spanning tree T , the tree is linearized via a depth-first visit. This gives a line graph S (the so-called spine of G) such that Φ S (y) ≤ 2 Φ T (y). By running a Nearest Neighbor (NN) predictor on S, [18] prove a mistake bound of the form Φ S (y) log n Φ S (y) + Φ S (y). As observed by [11] , similar techniques have been developed to solve low-congestion routing problems.. Another natural parametrization for the labels of a weighted graph that takes the graph structure into account is clusterability, i.e., the extent to which the graph nodes can be covered by a few balls of small resistance diameter. With this inductive bias in mind, [14] developed the Pounce algorithm, which can be seen as a combination of graph Perceptron and NN prediction. The number of mistakes has a bound of the form. where N (G, ρ) is the smallest number of balls of resistance diameter ρ it takes to cover the nodes of G. Note that the graph Perceptron bound is recovered when ρ = D W G . Moreover, observe that, unlike graph Perceptron's, bound (2) is never vacuous, as it holds uniformly for all covers of G (even the one made up of singletons, corresponding to ρ → 0). A further trick for the unweighted case proposed by [18] is to take advantage of both previous approaches (graph Perceptron and NN on line graphs) by building a binary tree on G. This "support tree" helps in keeping the diameter of G as small as possible, e.g., logarithmic in the number of nodes n. The resulting prediction algorithm is again a combination of a Perceptron-like algorithm and NN, and the corresponding number of mistakes is the minimum over two earlier bounds: a NN-based bound of the form Φ G (y)(log n) 2 and an unweighted version of bound (2) .. Generally speaking, clusterability and resistance-weighted cutsize E Φ T (y) exploit the graph structure in different ways. Consider, for instance, a barbell graph made up of two m-cliques joined by k unweighted φ-edges with no endpoints in common (hence k ≤ m).. 3 If m is much larger than k, then bound (2) scales linearly with k (the two balls in the cover correspond to the two m-cliques). On the other hand, E Φ T (y) tends to be constant: Because m is much larger than k, the probability of including any φ-edge in T tends to 1/k, as m increases and k stays constant. On the other hand, if k gets close to m the resistance diameter of the graph decreases, and (2) becomes a constant. In fact, one can show that when k = m even E Φ T (y) is a constant, independent of m. In particular, the probability that a φ-edge is included in the random spanning tree T is upper bounded by. , i.e., E Φ T (y) → 3 when m grows large. 4 When the graph at hand has a large diameter, e.g., an m-line graph connected to an m-clique (this is sometimes called a "lollipop" graph) the gap between the covering-based bound (2) and E Φ T (y) is magnified. Yet, it is fair to say that the bounds we are about to prove for our algorithm have an extra factor, beyond E Φ T (y), which is logarithmic in m. A similar logarithmic factor is achieved by the combined algorithm proposed in [18] .. An even more refined way of exploiting cluster structure and connectivity in graphs is contained in the paper of [15] , where the authors provide a comprehensive study of the application of dualnorm techniques to the prediction of weighted graphs, again with the goal of obtaining logarithmic performance guarantees on large diameter graphs. In order to trade-off the contribution of cutsize Φ W G and resistance diameter D W G , the authors develop a notion of p-norm resistance. The obtained bounds are dual norm versions of the covering ball bound (2) . Roughly speaking, one can select the dual norm parameter of the algorithm to obtain a logarithmic contribution from the resistance diameter at the cost of squaring the contribution due to the cutsize. This quadratic term can be further reduced if the graph is well connected. For instance, in the unweighted barbell graph mentioned above, selecting the norm appropriately leads to a bound which is constant even when k m. Further comments on the comparison between the results presented by [15] and the ones in our paper are postponed to the end of Section 5.. Departing from the online learning scenario, it is worth mentioning the significantly large literature on the general problem of learning the nodes of a graph in the train/test transductive setting: Many algorithms have been proposed, including the label-consistent mincut approach of [4, 5] and a number of other "energy minimization" methods -e.g., the ones by [31, 2] of which label propagation is an instance. See the work of [3] for a relatively recent survey on this subject.. Our graph prediction algorithm is based on a random spanning tree of the original graph. The problem of drawing a random spanning tree of an arbitrary graph has a long history -see, e.g., the recent monograph by [22] . In the unweighted case, a random spanning tree can be sampled with a random walk in expected time O(n ln n) for "most" graphs, as shown by [6] . Using the beautiful algorithm of [30] , the expected time reduces to O(n) -see also the work of [1] . However, all known techniques take expected time Θ(n The adversarial strategy. Numbers on edges are the probabilities p i,j of those edges being included in a random spanning tree for the weighted graph under consideration. Numbers within nodes denote the weight of that node based on the p i,j -see main text. We set the budget K to 6, hence the subset S contains the 6 nodes having smallest weight. The adversary assigns a random label to each node in S thus forcing |S|/2 mistakes in expectation. Then, it labels all nodes in V \ S with a unique label, chosen in such a way as to minimize the cutsize consistent with the labels previously assigned to the nodes of S.. experiments we tested a viable fast approximation where weights are disregarded when building the spanning tree, and only used at prediction time. Finally, the space complexity for generating a random spanning tree is always linear in the graph size.. To conclude this section, it is worth mentioning that, although we exploit random spanning trees to reduce the cutsize, similar approaches can also be used to approximate the cutsize of a weighted graph by sparsification -see, e.g., the work of [26] . However, because the resulting graphs are not as sparse as spanning trees, we do not currently see how to use those results. As explained in Section 4, WTA runs in two phases: (i) a random spanning tree is drawn; (ii) the tree is linearized and labels are sequentially predicted. As discussed in Subsection 1.1, Wilson's algorithm can draw a random spanning tree of "most" unweighted graphs in expected time O(n). The analysis of running times on weighted graphs is significantly more complex, and outside the scope of this paper. A naive implementation of WTA's second phase runs in time O(n log n) and requires linear memory space when operating on a tree with n nodes. We now describe how to implement the second phase to run in time O(n), i.e., in constant amortized time per prediction step.. Once the given tree T is linearized into an n-node line L, we initially traverse L from left to right. Call j 0 the left-most terminal node of L. During this traversal, the resistance distance d(j 0 , i) is incrementally computed for each node i in L. This makes it possible to calculate d(i, j) The line L has n = 27 nodes (the adjacent squares at the bottom). Shaded squares are the revealed nodes, connected through a dark grey doubly-linked list B. The depicted tree T has both unmarked (white) and marked (shaded) nodes. The arrows indicate the traversal operations performed by WTA when predicting the label of node i t : The upward traversal stops as soon as a marked ancestor anc(i t ) is found, and then a downward traversal begins. Note that WTA first descends to the left, and then keeps going right all the way down. Once i is determined, a single step within B suffices to determine i .. in constant time for any pair of nodes, since. On top of L, a complete binary tree T with 2 log 2 n leaves is constructed. 7 The k-th leftmost leaf (in the usual tree representation) of T is the k-th node in L (numbering the nodes of L from left to right). The algorithm maintains this data-structure in such a way that at time t: (i) the subsequence of leaves whose labels are revealed at time t are connected through a (bidirectional) list B, and (ii) all the ancestors in T of the leaves of B are marked. See Figure 4 .. When WTA is required to predict the label y it , the algorithm looks for the two closest revealed leaves i and i oppositely located in L with respect to i t . The above data structure supports this operation as follows. WTA starts from i t and goes upwards in T until the first marked ancestor anc(i t ) of i t is reached. During this upward traversal, the algorithm marks each internal node of T on the path connecting i t to anc(i t ). Then, WTA starts from anc(i t ) and goes downwards in order to find the leaf i ∈ B closest to i t . Note how the algorithm uses node marks for finding its way down: For instance, in Figure 4 the algorithm goes left since anc(i t ) was reached from below through the right child node, and then keeps right all the way down to i . Node i (if present) is then identified via the links in B. The two distances d(i t , i ) and d(i t , i ) are compared, and the closest node to i t within B is then determined. Finally, WTA updates the links of B by inserting i t between i and i .. In order to quantify the amortized time per trial, the key observation is that each internal node k of T gets visited only twice during upward traversals over the n trials: The first visit takes place when k gets marked for the first time, the second visit of k occurs when a subsequent upward visit also marks the other (unmarked) child of k. Once both of k's children are marked, we are guaranteed that no further upward visits to k will be performed. Since the preprocessing operations take O(n), this shows that the total running time over the n trials is linear in n, as anticipated. Note, however, that the worst-case time per trial is O(log n). For instance, on the very first trial T has to be traversed all the way up and down. This is the way we implemented WTA on the experiments described in the next section. We now present the results of an experimental comparison on a number of real-world weighted graphs from different domains: text categorization, optical character recognition, spam detection and bioinformatics. Although our theoretical analysis is for the sequential prediction model, all experiments are carried out using a more standard train-test scenario. This makes it easy to compare WTA against popular non-sequential baselines, such as Label Propagation. We compare our algorithm to the following other methods, intended as representatives of two different ways of coping with the graph prediction problem: global vs. local prediction.. Perceptron with Laplacian kernel. Introduced by [16] and here abbreviated as GPA (graph Perceptron algorithm). This algorithm sequentially predicts the nodes of a weighted graph G = (V, E) after mapping V via the linear kernel based on L + G + 1 1 , where L G is the laplacian matrix of G. Following [19] , we run GPA on a spanning tree T of the original graph. This is because a careful computation of the Laplacian pseudoinverse of a n-node tree takes time Θ(n + m 2 + mD) where m is the number of training examples plus the number of test examples (labels to predict), and D is the tree diameter -see the work of [19] for a proof of this fact. However, in most of our experiments m = n, implying a running time of Θ(n 2 ) for GPA. Note that GPA is a global approach, in that the graph topology affects, via the inverse Laplacian, the prediction on all nodes.. Weighted Majority Vote. Introduced here and abbreviated as WMV. Since the common underlying assumption to graph prediction algorithms is that adjacent nodes are labeled similarly, a very intuitive and fast algorithm for predicting the label of a node i is via a weighted majority vote on the available labels of the adjacent nodes. More precisely, WMV predicts using the sign of j : (i,j)∈E y j w i,j where y j = 0 if node j is not available in the training set. The overall time and space requirements are both of order Θ(|E|), since we need to read (at least once) the weights of all edges. WMV is also a local approach, in the sense that prediction at each node is only affected by the labels of adjacent nodes.. Label Propagation. Introduced by [31] and here abbreviated as LABPROP. This is a batch transductive learning method based on solving a (possibly sparse) linear system of equations which requires Θ(mn) time on an n-node graph with m edges. This bad scalability prevented us from carrying out comparative experiments on larger graphs of 10 6 or more nodes. Note that WMV can be viewed as a fast approximation of LABPROP.. In our experiments, we combined WTA and GPA with spanning trees generated in different ways (note that WMV and LABPROP do not use spanning trees).. Random Spanning Tree (RST). Each spanning tree is taken with probability proportional to the product of its edge weights -see, e.g., [22, Chapter 4] . In addition, we also tested WTA combined with RST generated by ignoring the edge weights (which were then restored before running WTA). This second approach gives a prediction algorithm whose total expected running time, including the generation of the spanning tree, is Θ(n) on most graphs. We abbreviate this spanning tree as NWRST (non-weighted RST).. Depth-first spanning tree (DFST). This spanning tree is created via the following randomized depth-first visit: A root is selected at random, then each newly visited node is chosen with probability proportional to the weights of the edges connecting the current vertex with the adjacent nodes that have not been visited yet. This spanning tree is faster to generate than RST, and can be viewed as an approximate version of RST.. Minimum Spanning Tree (MST). The spanning tree minimizing the sum of the resistors of all edges. This is the tree whose Laplacian best approximates the Laplacian of G according to the trace norm criterion -see, e.g., the paper of [19] .. Shortest Path Spanning Tree (SPST). [19] use the shortest path tree because it has a small diameter (at most twice the diameter of G). This allows them to better control the theoretical performance of GPA. We generated several shortest path spanning trees by choosing the root node at random, and then took the one with minimum diameter.. In order to check whether the information carried by the edge weight has predictive value for a nearest neighbor rule like WTA, we also performed a test by ignoring the edge weights during both the generation of the spanning tree and the running of WTA's nearest neighbor rule. This is essentially the algorithm analyzed by [18] , and we denote it by NWWTA (non-weighted WTA). We combined NWWTA with weighted and unweighted spanning trees. So, for instance, NWWTA+RST runs a 1-NN rule (NWWTA) that does not take edge weights into account (i.e., pretending that all weights are unitary) on a random spanning tree generated according to the actual edge weights. NWWTA+NWRST runs NWWTA on a random spanning tree that also disregars edge weights.. Finally, in order to make the classifications based on RST's more robust with respect to the variance associated with the random generation of the spanning tree, we also tested committees of RST's. For example, K*WTA+RST denotes the classifier obtained by drawing K RST's, running WTA on each one of them, and then aggegating the predictions of the K resulting classifiers via a majority vote. For our experiments we chose K = 7, 11, 17.. We ran our experiments on five real-world datasets:. RCV1. The first 10,000 documents 8 (in chronological order) of Reuters Corpus Volume 1, with TF-IDF preprocessing and Euclidean normalization. KROGAN. This is a high-throughput protein-protein interaction network for budding yeast. It has been used by [21] and [23] .. COMBINED. A second dataset from the work of [23] . It is a combination of three datasets: [12] 's, [20] 's, and [28] 's.

B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimization of nested SQL queries revisited Current methods of evaluatmg nested quenes m the SQL language can be mefficlent m a variety of query and data base. contexts F'revlous research m the area of nested queay opumlzatlon which sought methods of reducmg evaluation costs 18 summanzed, mcludmg a classuicatmn scheme for nested queries. elgonthms &sgned to transform each type of query tn a logically equ&ent form which may then be. evaluated more efficzntly, and a descnphon of a maps bug m one of these. algomhms Fmtber exanunation reveals another bug m the same algorithm Soluuons to these bugs are pmposed and mcorporated mto a new transfonnauon algontbm, and extenson.3 are pmposed wluch wdl albw the hmaformat~cm algonthms to handle a larger class of mcates A recursive algorithm for pmcessmg a geneml nested query IS presented and the actmn of ttus algorithm 1s demonstrated llus algorithm can be used to transform any nested quq SQL 1s a block-structured query language for data remeval and mampulanon developed at the IBM Research Laboratory m San Jose, Cahfomia [AST 751 SQL was mcorporated mto System R, the relational data base management system, also developed at the IBM San Jose Research Laboratory [AST 761 One of the most powerful features of SQL 1s the nesnng of query blocks For demonstration purposes, assume the followmg relahons S(SNO,SNAME,STATUS,CllY) -tbe Suppbers relauon P(PNO,PNAME,COLOR.WEIGHTCITY) -tbe Parts relauon SP(SNO,PNO.QTY,ORIGIN) -the. Shipment relauon. The pnmary keys for these relattons are SNO, PNO, and SNO,PNO respecttvely If we wanted the names of all suppliers who supply part P2 we could say PermIssIon to copy wlthout fee all or part of this mate& 1s granted provided that the copies are not made or dlstrlbuted for direct commercml advantage, the ACM copyright notice and the title of the pubhcatlon and Its date appear, and notlce IS given that copymg IS by permissIon of the Assoclatlon for Computmg Machmery To copy otherwise, or to republish, requires a fee and/or specfic permlssion 0 1987 ACM 0-89791-236-5/87/ooO5/0023 756. SELECT SNAME FROM s WHERE SNOIN (SELECT SNO FROM SP WHERE NO= 'pz'), This 1s an example of a query with a smgle level of nestmg The basic structure of a SQL query 1s a query block, which consists pnnclpally of a SELECT clause, a FROM clause, and zero or more WHERE clauses The first query block m a nested query 1s known as the outer query block and the next query block 1s known as the Inner query block The WHERE clause specifies the pdcates which the tuples remeved must sattsfy One type of predicate whzh can appear m the WHERE clause 1s a nesredpre&zfe, which 1s of the form [rzl Ck op QJ, where Q 1s a query block [KIM 82 4451 Q will always be a form of the SELECT statement The op may be a scalar or set membership operator A relation referred to in the mner query block shall be designated as an tnner relanon, and a relauon referred to m the outer query block shall be designated as an outer relunon Quenes can be nested to an arbitrary depth. In his 1982 paper "On Optmuzmg an SQL-hke Nested Query" [KIM 821, Won IGm showed that the conventional techmques used in implementing query nestmg, 1 e the techniques used in System R [SEL 79 333, can be very inefficient tables referenced m the inner query block of a nested query may have to be remeved once for each tuple of the relation referenced m the outer query block [KIM 82 4501 As a soluaon to this problem, Ktm proposed query transformation algonthms that would improve the efficiency of nested query evaluation, sometimes by orders of magmtude I-Its approach was to transform a nested query to a lo@cally equivalent smgle-level query (1 e wthout nestmg) this query could then be exammed by a query opttrmzer. such as that described m [SEL 791. for alternative methods of processing, mcludmg &fferent methods of performmg Joins To mtroduce am's results. his system of classification for nested quenes is outlmed below 2. ljpes of Nested Queries Won Ktm developed a classlficatton of nested query types, four of which are relevant to this paper They are described here bnefly for smgle-level nested quenes, as presented m [KIM 821 Fmd the part numbers of those parts whose quantmes on hand equal the number of shpments of those parts before l-l-80. Gwen the example tables PARTS and SUPPLY defined above, query 42 ~111 give the followmg result when evaluated using nested iteration This result Qffers from that obtamed usmg nested Iteration The reason why the transformation fads IS that m the formation of the temporary relatron, no tuples appear wtuch do not match the precllcates apphed to the mner relanon Thus, the COUNT function will never return zero, since the only groups It 1s apphed to are groups of tuples matchmg the precllcates Thus CT m the temporary relation ~11 never be zero I(lesshng explored a mal correction of the bug which mvolved ORmg a predicate to the WHERE clause of the transformed query m order to a postemm fmd where an empty set occurs to satisfy the pticate, but the mal correction fiuled on a query with more than one level of nestmg [KE 84 51 I(lesshng concludes that m attemptmg to use K~I's algorithm NEST-JA for transformmg type-JA nested quenes, " there seems to be no general way to recover values lost by COUNTS on a correlanon level greater than 1" [KIE 84 71 While thus does seem to be true m the context of the SQL language as specified m [AST 761, the problem can be solved If the outer Jam operation IS avadable m the processmg of the query where * is the special null value The outer Jam operaaon 1s unplemented m at least one commercial data base management system with which the authors are farmliar [ORA 861 To solve the COUNT bug an outer Join may be used in the creation of the temporary relation lesslmg's query 42 could be mmsforme4l to gve the followmg. Before lookmg at the result of this new query, let us look at the result of the outer Join between PARTS and SUPPLY ~th the condmons gwen m the creahon of the temporary relation TEMP3 The soluuons to the bugs described 111 the previous section suggest a modtfied algonthm for transformmg type-JA nested quenes, which shall be called algorithm NEST-JA2 Thus algonthm consists of three major parts Algorithm NEST-JA2 1 Project the Join column of the outer relation, and restnct it with any simple predtcates applymg to the outer relation 2 Create. a temporary relahon, Joimng the inner relaaon wtth the proJecaon of the outer relation If the aggregate function IS COUNT, the Join must be an outer JOUI, and the mner relation must be resmcted and proJected before the JOT 1s performed If the aggregate function is COUNT(*), compute the COUNT function over the JOT

O O O B B O O O O O O O O O B B O O O O O O O B O B I O O B O O O O O O O O O O O O B O O O O O O B O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O B O O O O O O O B O O O B O O O O O O O O B O O O B B O O O O O O O O O B O O O O O O O B O O O O O O O B I O O O O O O O O O B O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B B O O O O O O O O B B O O O O O O O O O O O O B O O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O