Doubly-Generalized LDPC Codes: Stability Bound over the BEC The iterative decoding threshold of low-density parity-check (LDPC) codes over the binary erasure channel (BEC) fulfills an upper bound depending only on the variable and check nodes with minimum distance 2. This bound is a consequence of the stability condition, and is here referred to as stability bound. In this paper, a stability bound over the BEC is developed for doubly-generalized LDPC codes, where the variable and the check nodes can be generic linear block codes, assuming maximum a posteriori erasure correction at each node. It is proved that in this generalized context as well the bound depends only on the variable and check component codes with minimum distance 2. A condition is also developed, namely the derivative matching condition, under which the bound is achieved with equality. LDPC codes [1] have been intensively studied in the last decade due to their capability to approach the Shannon limit under iterative belief-propagation decoding. An LDPC code of length N and dimension K can be graphically represented as a bipartite graph, known as Tanner graph, with N variable nodes (VNs) and M ≥ N − K check nodes (CNs) [2] . In the Tanner graph, the degree of either a VN or a CN is defined as the number of edges connected to it. A degree-n VN of an LDPC code can be interpreted as a length-n repetition code, i.e., as a (n, 1) linear block code repeating n times its only information bit towards the check node decoder (CND).. Instead, a degree-n CN of an LDPC code can be interpreted as a length-n single parity-check (SPC) code, i.e., as a (n, n − 1) linear block code.. An extension of the concept of LDPC code is represented by doubly-generalized LDPC (D-GLDPC) codes [3] , where the VNs and the CNs are allowed to be generic (n, k) linear block codes instead of repetition and SPC codes, respectively. If only the CND is generalized while all the VNs are repetition codes, then the code is said a generalized LDPC (GLDPC) code, or a Tanner code [2] .. In a D-GLDPC code the codes used as VNs and CNs are called component codes. In this work each component code is supposed to be a linear block code having a minimum distance d min ≥ 2.. The VNs and the CNs which are not repetition or SPC codes, are referred to as generalized nodes. The corresponding code structure is depicted in Fig. 1 . An (n, k) generalized VN is characterized by n connections towards the CND; moreover, k of the N D-GLDPC encoded bits are associated with the VN, and interpreted by the VN as its k information bits. Then, the codeword length of a D-GLDPC code with N V VNs is N = N V i=1 k i (k i being the dimension of the i-th VN). An (n, k) generalized CN is characterized by n connections towards the variable node decoder (VND), and is associated with n−k independent parity check equations. Then, the number of parity-check equations for a D-GLDPC code with N C CNs is M = N C i=1 (n i − k i ) (k i and n i being the dimension and length of the i-th check node, respectively). For a description of a D-GLDPC code iterative decoder over the AWGN channel and BEC we refer to [3] and [4] , respectively. For LDPC code ensembles, an important role is played by a theorem known as the stability condition [5] - [7] . The most important consequence of the stability condition is the possibility to upper bound the asymptotic iterative decoding threshold. If the communication channel is a BEC with erasure probability q, the stability condition leads to the following upper bound on the asymptotic threshold q * for the LDPC ensemble:. The inequality (1) is referred to as stability bound in this paper 1 . In (1), λ ′ (0) = λ 2 is the fraction 1 We use the nomenclature stability bound as, even though (1) is sometimes referred to as the stability condition, strictly speaking it is a consequence of it. For more details we refer to [7, Theorem 3 .66] and the related discussion.. of edges connected to the length-2 repetition VNs, while ρ ′ (1) is the derivative (computed at x = 1) of the LDPC CNs degree distribution ρ(x) = j≥2 ρ j x j−1 , where ρ j is the fraction of edges connected to SPC CNs of length j. The bound (1) was first developed from density evolution. Next we propose a simple graphical interpretation of (1) using extrinsic information transfer (EXIT) charts [8] . Let us denote by I A the average a priori mutual information in input to the VND or to the CND. Furthermore, let us denote by I E,V (I A , q) and I E,C (I A ) the average extrinsic information for the VND and CND respectively (these functions are usually referred to as EXIT functions). Then, (1) is equivalent to the following condition: for q = q * , the derivative of the VND EXIT function I E,V (I A , q), with respect to I A and evaluated at I A = 1, must be smaller than the derivative of the inverse CND EXIT function I. There exist LDPC degree distributions achieving the bound (1) with equality, so that their threshold over the BEC assumes the simple closed form q. distributions, the first occurrence of a tangency point between the VND EXIT function I E,V (I A , q) and the inverse CND EXIT function I. For LDPC code ensembles characterized by VNs and CNs with degree at least 2, the first equality is always satisfied as both terms are equal to 1. This occurs also for D-GLDPC codes with all variable and check component codes having a minimum distance d min ≥ 2 [9], which is an assumption of this paper. Then, only the second equality is considered in the sequel, and is referred to as the derivative matching condition.. In this paper, the stability upper bound (1) and the derivative matching condition are extended to D-GLDPC codes (and to GLDPC codes as a sub-case). Our derivations lead to the conclusion that only the check and variable component codes with minimum distance d min = 2, including length-2 repetition codes and SPC codes, contribute to the stability bound. We also show that for D-GLDPC codes satisfying the derivative matching condition the asymptotic threshold over the BEC can be expressed by a simple formula.. The paper is organized as follows. Some definitions and the notation used in the paper are introduced in Section II. In Section III the possibility to reduce the rank of a linear block code generator matrix by column elimination is discussed. Using these results, in Section IV and Section V the stability bound is developed for GLDPC codes and for D-GLDPC codes, respectively. Final remarks are given in Section VI. In this paper, a stability bound over the BEC has been developed for D-GLDPC codes. It Examples of such GLDPC code constructions are described in [12] - [16] . In general, increasing the fraction of strong generalized CNs can be very favorable from the point of view of the overall code minimum distance and then of the error floor, but presents drawbacks.. A first drawback is represented by an overall code rate loss which makes GLDPC codes with large fractions of strong generalized CNs of interest only for low or very low rate [17] .. The reason is briefly reviewed next. Let us consider a more general code structure, namely a D-GLDPC code. If we denote by r V, i and by r C,j the code rate of the type-i VNs and of the type-j CNs, respectively, the overall design rate is. which is monotonically increasing respect to any r V, i and to any r C,j . A generalized CN of length n has a code rate smaller than the code rate of a length-n SPC CN. Then, a large fraction of strong generalized CNs determines an overall rate loss. In GLDPC codes this rate loss is difficult to compensate even using large fractions of length-2 repetition VNs (which are the highest rate VNs available if all the node in the Tanner graph have minimum distance at least 2) so that usually the overall GLDPC code remains of low rate. A second drawback is that GLDPC codes with large fractions of strong generalized CNs and large fractions of length-2 repetition VN are typically characterized by a poor asymptotic threshold due to the large area gap between the EXIT curves in the EXIT function (see the Area Theorem in [18] ).. Allowing the generalization of the VND together with the generalization of the CND provides an increased flexibility in the code design, that can be exploited to overcome the above mentioned limitations. In particular, the rate loss due to the generalized CNs can be compensated using generalized VNs with a code rate larger than 1/2. In this context, a special class of generalized VNs is represented by (n, n − 1) SPC VNs each one having n edges towards the CND and associated with n − 1 encoded bits. It is shown in [19] that these codes can be effectively exploited for the design of D-GLDPC codes with good waterfall and error floor performance.. In this appendix, we develop the polynomial P i (·) defined in Theorem 4 for such VNs when represented in both systematic and cyclic form. We also propose a numerical example illustrating the capabilities offered by D-GLDPC codes with SPC VNs.

O O O O O O O O O O O O O B I I I O O B I I I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Lower bounds for linear locally decodable codes and private information retrieval Abstract. We prove that if a linear error-correcting code C : {0, 1} n → {0, 1} m is such that a bit of the message can be probabilistically reconstructed by looking at two entries of a corrupted codeword, then m = 2 Ω(n) . We also present several extensions of this result. We show a reduction from the complexity of one-round, informationtheoretic Private Information Retrieval Systems (with two servers) to Locally Decodable Codes, and conclude that if all the servers' answers are linear combinations of the database content, then t = Ω(n/2 a ), where t is the length of the user's query and a is the length of the servers' answers. Actually, 2 a can be replaced by O(a k ), where k is the number of bit locations in the answer that are actually inspected in the reconstruction. This paper is concerned with two related notions. The first notion is that of locally decodable codes (LDC), which are error-correcting codes that allow recovery of individual information bits based on a few (randomly selected) codeword bits. The second notion is that of private information retrieval (PIR) schemes, which are protocols allowing users to retrieve desired data items from several (non-colluding) servers without yielding any information to any individual server. The relation between these notions has been observed by some researchers before, and is further established in this paper.. The study of LDCs was initiated by Katz and Trevisan (2000) , who established superlinear (but at most quadratic) lower bounds on the length of 264 Goldreich et al. cc 15 (2006) codes that allow recovery based on a constant number of bits. In contrast, the best known constructions of LDCs supporting recovery based on reading k bits (Beimel et al. 2002) have length 2 n Θ(log log k/(k log k)) , which is of the form 2 n Ω(1) for every constant k. This leaves a huge gap between the known lower and upper bounds, and an important research goal is to try to close this gap. We take a first step in this direction by closing the gap (via improved lower bounds) for the special case of linear LDCs in which recovery is based on two bits. We note that all the known constructions of LDCs are linear codes, including constructions where the decoder makes more than two queries. See Trevisan (2004) for a survey on LDCs and their applications.. The study of PIR schemes was initiated by Chor et al. (1998) , who presented (among other schemes) a one-round, 2-server PIR scheme of communication complexity O(n 1/3 ). The question of whether their (2-server) PIR scheme has the lowest communication complexity possible has been open since. We present several results that are related to this question; all our results relate to the special case of one-round, 2-server PIR schemes in which the servers' answers are always linear combinations of the data bits. Again, we note that in all the known constructions of PIR schemes the server's answers are linear combinations of the data bits. This is also true of schemes involving more than two servers. See Gasarch (2004) for a survey on PIR schemes.

B I O O B I I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fully Dynamic Maintenance of k-Connectivity in Parallel AbstractÐGiven a graph q Y i with n vertices and m edges, the k-connectivity of q denotes either the k-edge connectivity or the k-vertex connectivity of q. In this paper, we deal with the fully dynamic maintenance of k-connectivity of q in the parallel setting for k PY Q. We study the problem of maintaining k-edge/vertex connected components of a graph undergoing repeatedly dynamic updates, such as edge insertions and deletions, and answering the query of whether two vertices are included in the same k-edge/ vertex connected component. Our major results are the following: 1) An NC algorithm for the 2-edge connectivity problem is proposed, which runs in ylog n logman time using yn QaR processors per update and query.2) It is shown that the biconnectivity problem can be solved in ylog P n time using ynPnY na log n processors per update and yI time with a single processor per query or in ylog n log m n time using ynPnY na log n processors per update and ylog n time using ynPnY na log n processors per query, where XY X is the inverse of Ackermann's function. 3) An NC algorithm for the triconnectivity problem is also derived, which takes ylog n log m n log n log log naQnY n time using ynQnY na log n processors per update and yI time with a single processor per query. 4) An NC algorithm for the 3-edge connectivity problem is obtained, which has the same time and processor complexities as the algorithm for the triconnectivity problem. To the best of our knowledge, the proposed algorithms are the first NC algorithms for the problems using yn processors in contrast to m processors for solving them from scratch. In particular, the proposed NC algorithm for the 2-edge connectivity problem uses only yn QaR processors. All the proposed algorithms run on a CRCW PRAM.Index TermsÐNC algorithms, 2-edge/vertex connectivity, 3-edge/vertex connectivity, dynamic data structures, parallel algorithm design and analysis, graph problems. A fully dynamic graph algorithm is one that allows edge insertions and deletions and recomputes a desired graph property quickly after each such update. A partially dynamic graph algorithm is one that allows edge insertion updates only. In this paper, we focus on the design of fully dynamic NC algorithms for maintaining the k-connectivity of graphs. Given a graph q Y i with n vertices and m edges, the k-connectivity of q refers to either the k-edge connectivity or k-vertex connectivity [18] , which will be formally defined in Section 2.1. We present our results in the following order: First, we decompose q into a number of disjoint connected subgraphs and build data structures for each of them to keep the information on the subgraph. Also, we build a data structure to keep the information among the subgraphs, which is referred to as the compact representation q of q. q is the super graph of q, defined as follows: Each vertex in q corresponds to a subgraph of q. There is an edge between two vertices in q if there are edges in q between the vertices in the two corresponding subgraphs. For any two subgraphs in q, we use two data structures to keep the edge information between them. Then, we assume that all the data structures for 2-edge connected components in each subgraph have been already built. Now, consider a query which asks whether two given vertices x and y are in the same 2-edge connected component in q. To answer this query, we proceed as follows: Let x be in g x and y be in g y . We check whether g x and g y are in the same 2-edge connected component in q. If yes, then x and y are in the same 2-edge connected component in q. Otherwise, for every articulation point g in q separating g x from g y , we examine the induced subgraph of q by the vertices in g to see whether there is a potential bridge internal to the subgraph separating x from y in q. If yes, then x and y are not in the same 2-edge connected component in q. Finally, we deal with the fully dynamic maintenance of the data structures when there is an edge insertion or deletion. Note that each insertion/deletion only involves updating a constant number of subgraphs and q. Accordingly, we only update those involved subgraphs and q instead of the entire graph q, which is much cheaper in terms of the maintenance cost because of the smaller sizes of these subgraphs and q compared to q. Therefore, it is possible to reduce the maintenance cost to respond the updates and queries.. The rest of this section is organized as follows: Section 3.2 introduces the data structures that the proposed algorithm uses and Section 3.3 presents algorithms for constructing the data structures. Section 3.4 proposes an algorithm for query processing, using the proposed data structures. Section 3.5 shows how to maintain the data structures efficiently when there are edge insertions and deletions. In this paper, we have studied the problem of fully dynamic maintenance of k-connectivity of graphs and presented the first NC algorithms withỹn work for it, where k PY Q. In particular, our NC algorithm for 2-edge connectivity uses on processors only. For the biconnectivity problem, we have obtained an NC algorithm with yn processors. From this algorithm, we have also derived NC algorithms with yn processors for the fully dynamic maintenance of 3-edge connectivity and triconnectivity. However, for the addressed problems there is still a gap between the best sequential time complexity for them and the amount of work needed in parallel. It is interesting and challenging to find better NC algorithms for them to reduce or eliminate the gap.. Weifa Liang (M '99-SM '01)received the PhD degree from the Australian National University in 1998, the ME degree from the University of Science and Technology of China in 1989, and the BSc degree from Wuhan University, China in 1984, all in computer science. He is currently a lecturer in the Department of Computer Science at the Australian National University. His research interests include routing protocol design for high speed networks, design and analysis of parallel and distributed algorithms, data warehousing and OLAP, query optimization, and graph theory. He is a senior member of the IEEE.. Richard P. Brent (M'72-SM'83-F'91) received the BSc (hons) degree in mathematics from Monash University, Australia in 1968 and the PhD degree in computer science from Stanford University, California in 1971. Since 1998, he has been a professor of computing science at Oxford University, England. His research interests include parallel computer architectures, analysis of algorithms (especially parallel algorithms), numerical analysis, and computational number theory. He is a member of AMS, SIAM, Sigma Xi, and a fellow of the ACM and of the IEEE.. Hong Shen is a professor of computer science and research director of the Parallel Computing Unit in the School of Computing and Information Technology, Griffith University, Australia. He has published extensively in the areas of parallel and distributed computing, high-performance networking, algorithms, and data mining. He has served as editor, associate editor, and editorial board member of five international journals and chaired several international conferences.. F For more information on this or any computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I I B I I I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


SMS-based web search for low-end mobile devices Short Messaging Service (SMS) based mobile information services have become increasingly common around the world, especially in emerging regions among users with low-end mobile devices. This paper presents the design and implementation of SMSFind, an SMS-based search system that enables users to obtain extremely concise (one SMS message of 140 bytes) and appropriate search responses for queries across arbitrary topics in one round of interaction. SMSFind is designed to complement existing SMS-based search services that are either limited in the topics they recognize or involve a human in the loop.Given an unstructured search query, SMSFind, uses a conventional search engine as a back-end to elicit several search responses and uses a combination of information retrieval techniques to extract the most appropriate 140-byte snippet as the final SMS search response. We show that SMSFind returns appropriate responses for 57.3% of ChaCha search queries in our test set; this accuracy rate is high given that ChaCha employs a human to answer the same questions. We have also deployed a pilot version of SMSFind for use with a small focus group in Kenya to explore the interaction issues of such a system and share our early experiences. The exceptional growth of the mobile phone market has motivated the design of new forms of mobile information services. With the growth of Twitter [1] , SMS GupShup [2] and other social messaging networks, the past few years have witnessed a growing prevalence of Short-Messaging Service (SMS) based applications and services. SMS-based services are also increasingly common in developing regions. Despite the increasing power of mobile devices with the advent of "smart phones", a significant fraction of mobile devices in developing regions are still simple low-cost devices with limited processing and communication capabilities. Due to a combination of social and economic factors, voice and SMS will likely continue to remain the primary communication channels available for a non-trivial fraction of the population in developing regions.. For any SMS-based web service, efficient SMS-based search is an essential building block. SMS-based search is a rapidly growing global market with over 12 million subscribers as of July 2008 [3] . An SMS message is constrained to 140 bytes which drastically limits the amount of information in a search response. SMS-based search is also non-interactive due to the search response time; anecdotally [4] , existing SMS-based search engines take on the order of tens of seconds [5, 6] to several minutes per response [7] . Even without the 140-byte SMS size constraint, tailoring traditional web search to mobile devices is a challenging problem due to the small form factor and low bandwidth. Unlike desktop search, users on mobile devices rarely have the luxury of iteratively refining search queries or sifting through pages of results for the information they want [8] . In this paper, we address the problem of SMS-based search: how does a mobile user efficiently search the Web using one round of interaction where the search response is restricted to one SMS message?. Though we do not know the internals of existing SMS search algorithms, we can observe from the user interface and documentation that existing automated services for SMS web search such as Google SMS [5] and Yahoo! oneSearch [6] ) encourage users to enter queries for a number of pre-defined topics, or verticals. These pre-defined topics are either identified through the use of special keywords within the search query such as "define" or "movies" (e.g. Google SMS: "define boils") or have specialized parsers to determine which of the topics is intended (e.g. querying "AAPL" to Yahoo! oneSearch is a query for information about "stock quote"). ChaCha [7] , a recent SMS-based search engine, hires humans to search the web and answer questions in an SMS response. TradeNet, now called eSoko [9] , is a mobile marketplace platform in Ghana which would require an SMS based search service as well. Instant access to small bits of information is the motivation behind all of the existing SMS based search systems and of this work.. None of the existing automated SMS search services is a complete solution for search queries across arbitrary topics. Similar to traditional web search queries, SMS search queries suffer from the long tail phenomenon: there exists a long tail of search queries whose topics are not popular (e.g. "what are graduation gift ideas?" or "what chemicals are in a fire extinguisher"). We confirm that this indeed is the case in our sample of ChaCha questions where only 21% of the queries in our data set are verticals and 79% are long tailed.. In this paper, we describe the design and implementation of SMSFind, an SMS-based search engine specifically designed for the long tail of search queries that are spread across a wide range of topics. These topics represent the queries in a mobile search workload that are not answered by existing domain specific verticals. SMSFind is designed to integrate into an existing SMS search service to answer queries for unsupported long tail topics. Given a query, SMSFind uses a traditional search engine as a back-end to elicit several search results and extract the appropriate 140 bytes as the SMS search response. Section 6.2 further explains how vertical and long tail queries are defined in this work.. SMSFind uses a combination of well-known information retrieval techniques to address the appropriate information extraction problem. SMSFind is designed for unstructured queries supporting a similar format as a standard search engine query. The key idea of SMSFind is that meaningful SMS queries typically contain a term or a collection of consecutive terms in a query that provides a hint as to what the user is looking for. The hint for a query can either be explicitly provided by the user or automatically derived from the query. SMSFind uses this hint to address the information extraction problem as follows: Given the top N search responses to a query from a search engine, SMSFind extracts snippets of text from within the neighborhood of the hint in each response page. SMSFind scores snippets and ranks them across a variety of metrics.. We have evaluated SMSFind on a corpus of questions screen gathered from the website of the ChaCha SMS question/answering service. The ChaCha search queries are in question-style format which we converted to unstructured query format using a set of transformation rules. Based on extensive evaluation and human verification, we show that SMSFind can answer 57.3% of ChaCha queries. This is a significant result because these questions are currently being answered manually by humans. There are still questions that are difficult to answer using statistical methods such as, "do they kill horses to make glue" (as we discuss in Section 7.2), but to the best of our knowledge this is the first effort that addresses the problem of SMS-based search for long tail mobile queries. [10] appears to be working toward this as well, but the technical approach and how it relates to SMS search is unclear from the publicly available information at this time. There has been little work on SMS-based search for arbitrary topics due to, until recently, the initial lack of a welldefined business case. The explosive growth in prevalence of affordable low-end mobile devices throughout the world has created a large market for mobile information services. Since mobile users in many parts of the world use low-end mobile devices with SMS as their primary data transport, SMS-based search becomes a critical problem to address on the path to enabling SMS-based services.. In this paper, we have presented SMSFind, an automated SMS-based search response system that is tailored to work across arbitrary topics. We find that a combination of simple Information Retrieval algorithms in conjunction with existing search engines can provide reasonably accurate search responses for SMS queries. Using queries across arbitrary topics from a real-world SMS question/answering service with human-in-the-loop responses, we show that SMSFind is able to answer 57.3% of the queries in our test set. Although more powerful IR and NLP techniques are bound to improve performance, this work represents a foray into an open and practical research domain. The design of our system and the evaluation was motivated by the SMS search problem. Our goal was to understand how existing techniques would perform in this problem domain and not to supplant or reinvent the vast amounts of research in the Information Retrieval (IR) space. While there has been relatively little research on SMSbased search, in this section, we take a slightly larger view of the problem space and contrast SMSFind with mobile search services, question/answering (Q/A) systems from the Text REtrieval Conference (TREC) community [11] , and text summarization techniques. The core SMSFind algorithm is implemented in only 600 lines of Python code and uses publicly available parsing libraries. We have not implemented any optimizations or caching, but SMSFind generally returns results within 5-10 seconds while running on a 1.8Ghz Duo Core Intel PC with 2 GB of RAM and a 2 Mbps broadband connection. This response time is dominated by the time required to fetch query results from Google and download web pages referred to by the results. To deploy SMSFind as a search service we implemented a front-end to send and receive SMS messages. We setup a SMS short code with a local telco in Kenya, and route all SMS requests and responses to and from our server machine running the service across a Samba 75 GSM modem and Kannel an open source SMS Gateway [34] . As a proof of concept, and to improve the overall usability of our system we have also implemented interfaces to several basic verticals as a part of the service including: weather, definitions, local business results, and news. Each of these interfaces to verticals is under 150 lines of Python code. We are still iterating with our focus group on the implementation details of our system.

O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O O B O O O O O O O O O O B B O O O O O O O O O O O O O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O B B O O O O O O B O O O O O O O O B O B B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A theoretical model for template-free synthesis of long DNA sequence Abstract This theoretical scheme is intended to formulate a potential method for high fidelity synthesis of Nucleic Acid molecules towards a few thousand bases using an enzyme system. Terminal Deoxyribonucleotidyl Transferase, which adds a nucleotide to the 3 0 OH end of a Nucleic Acid molecule, may be used in combination with a controlled method for nucleotide addition and degradation, to synthesize a predefined Nucleic Acid sequence. A pH control system is suggested to regulate the sequential activity switching of different enzymes in the synthetic scheme. Current practice of synthetic biology is cumbersome, expensive and often error prone owing to the dependence on the ligation of short oligonucleotides to fabricate functional genetic parts. The projected scheme is likely to render synthetic genomics appreciably convenient and economic by providing longer DNA molecules to start with. To a great extent the progress in biotechnology and basic biomedical research has been centralized on the major advances in DNA synthesis and sequencing. Elucidation of the genetic code (Khorana 1968) , production of synthetic gene (Agarwal et al. 1974) , widespread use of PCR (Kleppe et al. 1971; Saiki et al. 1988) , sequencing of the human genome (Venter et al. 2001; Lander et al. 2001) , and the synthesis of whole genome of a microorganism (Gibson et al. 2008) are powerful examples. These applications along with many others have pivotally depended on the ability to synthesize short oligonucleotides, used as primers, typically ssDNA 10-80 base in length (Caruthers 1985) . The widespread application of synthetic biology is essentially limited by the complexity and cost of assembling short oligonucleotides into longer functional DNA (Endy 2005) . In addition to whole genomes (Hutchison et al. 1999; Smith et al. 2003; Gibson et al. 2008) , construction of entire biochemical pathways Mehl et al. 2003; Kodumal et al. 2004 ) and genetic circuitry (Elowitz and Leibler 2000; Sprinzak and Elowitz 2005) are illustrative of synthetic biology requiring synthesis of far more than a single gene. Therefore, the rapid availability of predefined DNA, more than 1 Mb in length at a cost per base comparable to or less than oligonucleotides is greatly appealing. Despite important progress in this direction through building large numbers of genes by harnessing the massively parallel form of oligonucleotide synthesis to produce microarrays (Richmond et al. 2004; Tian et al. 2004) , still the procedures depend on the chemical synthesis of short oligos. Thus the availability of longer DNA molecules is in principle restricted by the inherent limit of the chemical processes for oligonucleotide synthesis.. The error rate of synthesizing oligonucleotides is also of immense importance. At an error rate of 1 in 600 bp, it is required to sequence 10 clones to get a DNA construct of 1 Kb and 100 clones for 2 Kb (Baedeker and Schulz 1999; Withers-Martinez et al. 1999; Hoover and Lubkowski 2002; Chalmers and Curnow 2001) . Thus large target fabrication is extremely difficult and error prone at this rate (Cello et al. 2002) . The error was reduced to 1 in 1,400 bp by Tian et al. (2004) using microchip based multiplex synthesis. Protein mediated error correction has been utilized to reduce the rate further to 1 in 10,000 bp (Carr et al. 2004) .. If the starting material is precise DNA molecules 1-10 Kb in length rather than *50 bases, the speed and over all efficiency of synthetic genomics would tremendously increase because synthesis of longer constructs would be possible with fewer clones and sequencing. An extensive application of synthetic genes would be possible with a more convenient and reliable process for their fabrication. Thus an investigation for an efficient synthetic scheme is worth pursuing.. An attempt to delineate the necessary properties of a successful long DNA synthesis process leads to some salient features. Firstly, the error rate should be less than 10 -4 in order to allow convenient synthesis of DNA constructs around 10 5 bp in length. Second, the maximum synthesizable length of DNA should be greater than 10 5 as current protocols allow transformation with DNA constructs of up to 3 9 10 5 -20 9 10 5 bp in length (Glick and Pasternak 2003) . These features indicate that preferably an enzymatic system might be successful. Moreover, enzymatic systems would in principle be more specific and efficient than chemical processes (Sitnitsky 2006) . However, the enzymes those can synthesize DNA without templates are not controlled. For example TdT will add a homopolymeric tract of nucleotides when provided with a primer and a single type of NTP (Bollum 1978; Ratliff 1981) . When provided with a mixture of nucleotides and a primer, TdT act as a random-sequence generator (Bollum 1978; Ratliff 1981) . The physiological role of TdT conform to this random sequence generation as it provides additional variations in hematopoetic cells through acting as a somatic mutator, diversifying the amino acid sequence in the variable region of immunoglobulin molecules (Ratliff 1981) .. TdT has already been utilized in genomics. It has been used for the production of synthetic homo and heteropolymers (Bollum 1974) , homopolymeric tailing of linear duplex DNA (Deng and Wu 1983; Eschenfeldt et al. 1987) , oligodeoxyribonucleotide and DNA labeling (Deng and Wu 1983; Tu and Cohen 1980; Vincent et al. 1982; Kumar et al. 1988; Igloi and Schiefermayr 1993) , rapid amplification of cDNA ends (Frohman et al. 1988 ) and in situ localization of apoptosis (Gorczyca et al. 1993 ). Fukuoka (1992, 1997) used TdT to add homopolymeric oligo dC tract to 3 0 end of ss linear DNA in order to facilitate synthesis of ds linear DNA using oligo dG primer. However, no attempt to use TdT to synthesize a defined DNA sequence has been reported. Perhaps the random sequence generation by TdT in an uncontrolled system has prevented this.. Here a theoretical model for predefined long DNA synthesis based on TdT is proposed. The scheme depends on the addition of 3 0 AcdNTP to the 3 0 -OH end of a DNA. The Ac group is suggested to prevent polymer formation and thus prevent the chain extension to a single base in each cycle (Fig. 1) . After extension the pH of the system is decreased to activate AE and AP while TdT would be deactivated. AP would hydrolyze excess 3 0 AcdNTP while AE would remove the Ac groups. Then dialysis would render the small molecules out of the system. Dialysis would also elevate the pH to activate TdT and deactivate AE and AP. Thus a new cycle of extension can be initiated by addition of 3 0 AcdNTP. Theoretical analysis based on available data suggests the scheme to be highly promising in synthesizing long DNA molecules. The scheme proposed here is a potentially efficient enzymatic method for de novo DNA synthesis. From available data on the thermodynamics of the reactions and properties of the involved enzymes, the scheme is theoretically feasible. However, experimental demonstration would be crucial for its practical application.. A major difficulty may be the low turnover number of TdT with 3 0 AcdNTP as substrate. From the reported results of incorporation of modified bases by TdT (Deng and Wu 1983; Tu and Cohen 1980; Vincent et al. 1982; Kumar et al. 1988; Igloi and Schiefermayr 1993) , it is inferred that TdT would accommodate an acetyl group. Recombinant enzymes are likely to be available to circumvent the problem in case native TdT does not incorporate 3 0 AcdNTP. Recombinant enzymes may also lead to a TdT with higher affinity to primers further limiting the error rate.. Nanobioreactors with capabilities of rapid change in pH would offer great advantage not only for this scheme but also for synthetic methods requiring sequential temporal activity switching of different sets of enzymes. Enzymatic correction (Carr et al. 2004 ) may be used on the longer DNA sequences. This combination may limit the error further down to 1 in 150,000. The availability of longer DNA for gene assembly would significantly increase the power and range of synthetic genomics. Not only small microorganism genomes, large genomes of higher organisms may also become amenable to synthesis with longer starting material. The study on enzyme properties and possible combinations suggests that an enzymatic synthesis of predefined long DNA molecule is theoretically plausible. Based on the available data the process depicted in Fig. 3 is expected to be successful. AcOH would be added to decrease the pH to 5.5 while dialysis in a buffer of pH 7.5 would serve multiple functions, including removal of AcOH, phosphate groups, nucleosides and other small molecules in addition to restoring the pH to 7.5.. Incubation time T1 would depend on the relative concentration of TdT, primer, 3 0 AcdNTP and ID. The total time required for completing a step of the synthetic scheme would be determined according to the following equation:. t step ¼ time required for pH switching þ time required for reaction completion. where, a is radius of the reactor, In order to complete pH switching within less than 0.1 s the radius of the reactor should be less than 3:94 Â 10 À3 cm with a volume less than 1:92 Â 10 À7 ml or 0:19 nl: As an enzyme catalyzed reaction is designed to take place in nano liter volume, the reactor is named nanobioreactor. Use of nanobioreactors would greatly reduce the time of pH switching and thus lead to faster synthesis.. Incubation time T2 would depend on the size of the reactor, K cat [AP] and K cat [AE] . The smaller the reactor the less time would be necessary to activate AE and AP through shift in pH. Moreover, a slow pH shift may lead to a transitory period with both TdT, ID and AP, AE enzyme pairs active; thus increase error rate. Therefore a rapid pH change essential. Higher enzyme activity would decrease the time required for complete hydrolysis of excess bases and deprotection of 3 0 -OH groups. Incubation time T3 would depend mostly on the time required to restore the pH back to 7.5. Thus the size of the reactor would again be an important factor. Number of possible cycles N would depend on the error rate, which would in turn depend on the completeness of reaction at each step. As reaction of each step in the cycle is thermodynamically favorable, the completeness is in principle warranted, provided with sufficient time. Since the K M of TdT for oligonucleotide primer is in the range of 10 -4 , at equivalent molar ratio with primer TdT is expected to make less than 1 in 10,000 errors. The error rate may however increase if incubation time T1 is longer due to slow pH shift. Thus faster pH shift would maintain high fidelity. The scheme Synthesis of a predefined DNA sequence requires the addition of a single known nucleotide at each step to a given nucleic acid polymer. Thus the problem is essentially reduced to formulating a system such that:. is allowed, while:. is forbidden. Also the allowed reaction should be virtually complete such that no unextended DNA can go through the second round. Searching for enzymes able to extend DNA molecules in the BRENDA enzyme database (Schomburg et al. 2002 (Schomburg et al. , 2004 Barthelmes et al. 2007 ) TdT (EC: 2.7.7.31) was selected to be a potential enzyme because it can add nucleotides to the 3 0 -OH group of DNA several Kb long and can add modified bases.. In order to prevent formation of a homopolymeric tract the 3 0 -OH group of the incoming nucleotide should be blocked. Methyl and acetyl groups are extensively used as OH protecting groups. Due to the promising convenience of deprotection by acetylesterase (EC: 3.1.1.6); acetyl group is expected to be a better candidate. Thus, instead of dNTP the DNA primer is to be elongated by addition of a 3 0 AcdNTP catalyzed by TdT. There is lack of K M and K cat data of TdT with 3 0 AcdNTP. Nevertheless, the accommodation of modified bases by TdT indicates a high possibility of addition of a single 3 0 AcdNTP to 3 0 -OH end of a DNA. Now the completion of the reaction remains a problem. TdT has K m value of 3 9 10 -4 mM for oligonucleotide primers at pH 8.2 in the presence of Mn ?2 (Coleman 1977 ) and turnover number 0.833 for ATP (Bollum 1974) . Thus TdT should be present in a high molar ratio with respect to primers in order to ensure the completeness of the reaction. Further, the polymerization reaction would be energetically favorable as pyrophosphate may be quickly removed by hydrolysis.. A new cycle of base addition must be preceded by removal of excess 3 0 AcdNTP and deprotection of the 3 0 OH group of the extended DNA. Acid phosphatase (EC: 3.1.3.2) is a suitable candidate for hydrolysis of excess 3 0 AcdNTP while acetylesterase is preferred for deprotection.. However the process raises problem regarding precise temporal order. As depicted in Fig. 2 , if the protective Ac group is removed before complete hydrolysis of 3 0 AcdNTP, additional bases may add to the extended DNA. Moreover AE deacetylates wide range substrates. Thus 3 0 AcdNTP may become deacetylated before incorporation into DNA. This event would also fail the objective. Thus TdT must be inactive when AE is active and vice versa. Bovine TdT has a pH optimum at 7.5 and has limited activity below pH 6.9 (Coleman 1977) however it is stable at pH down to 4.5. On the other hand Aspergillus niger AE has a pH optimum of 5.5 with limited activity over pH 6 (Kormelink et al. 1993) . A. niger AE is however stable at pH 8. Therefore, the elongation step should be catalyzed by TdT at pH 7.5 while the deprotection step should take place at pH 5.5.. Penicillium chryrsogenum acid phosphatase has pH optimum at pH 5.5 (Haas et al. 1991) . Therefore it is expected to be an efficient scavenger of excess 3 0 AcdNTP after elongation. E. coli inorganic diphosphatase (EC: 3.6.1.1) has pH optimum at pH 7.5 (Vainonen et al. 2005) . Thus it can be utilized to hydrolyze pyrophosphate formed during elongation by TdT. The rapid change is pH demanded by such a pH controlled system would be very difficult to achieve in a traditional reactor system. A nanoreactor would provide distinctive advantage of rapid condition alteration. Thus, the enzyme system in a nanoreactor coupled with pH regulation is a promising one for template free long DNA synthesis.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I


Constraint Satisfaction Techniques in Planning and Scheduling Abstract Over the last few years constraint satisfaction, planning, and scheduling have received increased attention, and substantial effort has been invested in exploiting constraint satisfaction techniques when solving real life planning and scheduling problems. Constraint satisfaction is the process of finding a solution to a set of constraints. Planning is the process of finding a sequence of actions that transfer the world from some initial state to a desired state. Scheduling is the problem of assigning a set of tasks to a set of resources subject to a set of constraints. In this paper, we introduce the main definitions and techniques of constraint satisfaction, planning and scheduling from the Artificial Intelligence point of view. Planning and scheduling techniques have recently seen important advances thanks to the application of constraint satisfaction models and tools. Most real-world problems can be cast as highly coupled planning and scheduling problems, where resources must be allocated so as to optimize overall performance objectives. Therefore, solving these problems requires an adequate mixture of planning, scheduling and resource allocation to competing goal activities over time in the presence of complex state-dependent constraints. Solutions to these problems must integrate resource allocation and plan synthesis capabilities, which can be efficiently managed by using constraint satisfaction techniques. The aim of this paper is to give a general overview of three different but interrelated areas: constraint satisfaction, planning, and scheduling.

B I O O B O B O O O O O O B I B O B O O O O O O O O O O O O B I O O O O O B O B O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O B O O O O O O O B O B O O O O O O O O O O O B I O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I B O B


Affinity-Based Network Interfaces for Efficient Communication on Multicore Architectures Abstract Improving the network interface performance is needed by the demand of applications with high communication requirements (for example, some multimedia, real-time, and high-performance computing applications), and the availability of network links providing multiple gigabits per second bandwidths that could require many processor cycles for communication tasks. Multicore architectures, the current trend in the microprocessor development to cope with the difficulties to further increase clock frequencies and microarchitecture efficiencies, provide new opportunities to exploit the parallelism available in the nodes for designing efficient communication architectures. Nevertheless, although present OS network stacks include multiple threads that make it possible to execute network tasks concurrently in the kernel, the implementations of packet-based or connection-based parallelism are not trivial as they have to take into account issues related with the cost of synchronization in the access to shared resources and the efficient use of caches. Therefore, a common trend in many recent researches on this topic is to assign network interrupts and the corresponding protocol and network application processing to the same core, as with this affinity scheduling it would be possible to reduce the contention for shared resources and the cache misses. In this paper we propose and analyze several configurations to distribute the network interface among the different cores available in the server. These alternatives have been devised according to the affinity of the corresponding communication tasks with the location (proximity to the memories where the different data structures are stored) and characteristics of the processing core. As this approach uses several cores to accelerate the communication path of a given connection, it can be seen as complementary to those that consider several cores to simultaneously process packets belonging to either the same or different connections. Message passing interface (MPI) workloads and dynamic web servers have been considered as applications to evaluate and compare the communication performance of these alternatives. In our experiments, performed by full-system simulation, improvements of up to 35% in the throughput and up to 23% in the latency have been observed in MPI workloads, and up to 100% in the throughput, up to 500% in the response time, and up to 82% in the requests attended per second have been measured in dynamic web servers. The availability of high bandwidth links (multigigabit Ethernet, Infiniband, etc.) [1] , along with the rate of link bandwidth improvements can shift the communication bottleneck towards the servers. Thus, the network interface (NI) of the server is an important element in the communication path, and optimized NI architectures to reduce the corresponding communication overhead due to context switching, multiple data copies, and interrupt mechanisms have constituted a relevant researching line in the years. After first approaches such as light communication protocols and user-level network interfaces [2] , the presence of multiple processors in the computing nodes, including the emergence of multicore processor, motivated proposals towards dedicating some processors of the node to executing the communication tasks, such as the previously proposed protocol offloading and onloading approaches [3] [4] [5] [6] , and network stack parallelization strategies to simultaneously process packets from the same or different connections [7] [8] . Offloading consists in the use of processors included in the network interface cards (NICs) for protocol processing. In this case, the NIC can directly interact with the network without the host CPU participation, thus allowing not only to decrease the CPU overhead for interrupt processing, but also a protocol latency reduction for short control messages (such as ACKs) that, in this way, they would not have to go to the main memory through the I/O bus. This I/O traffic reduction contributes to avoiding the bus contention and it is also possible to improve the efficiency of the direct memory access (DMA) from the NIC, whenever short messages are assembled to generate less DMA transferences. Nevertheless, some work [9] argued that protocol offloading does not clearly benefit the communication performance of the applications because 1) the NIC CPU speed must be high enough to process the protocol as fast as not to become a communication bottleneck, and moreover, 2) the API for communicating the NIC (that executes the offloaded protocol) and the host CPU (that executes the OS, the application and the API), could be very complex, specially for the TCP stack [9] . Another proposed alternative to release host CPU cycles is the so-called protocol onloading or, more specifically, TCP onloading [10] . This technique proposes the use of a general-purpose processor in a multicore microprocessor, or in an symmetric multi-processor (SMP), for protocol processing. With onloading, the processor that executes the communication software usually has the same speed and characteristics as the host CPU, and it can access the main memory with the same rights as the host CPU. Although this alternative exploits the current trend towards multicore architectures or multiprocessor nodes, onloading the communication tasks in a multicore platform could result in an overload on memory access subsystem during intensive communication workloads, thus degrading the overall application performance. There are some papers that compare both techniques [11] and show that the relative improvement on peak throughput offered by offloading and onloading depends on the rate of application workload to communication overhead, the message sizes, and other characteristics of system architecture, such as the I/O bandwidths and the way the NIC is connected to the system processor and memory. Moreover, improvements such as the reduction in the number of interrupts, the DMA transfer optimizations, and the use of mechanisms to avoid bottlenecks in the receiverside such as split headers, asynchronous copy by using DMA, and multiple receive queues [13] could be implemented by both onloaded or offloaded approaches. Presently, the availability of multicore processors with more than four cores has opened new possibilities. More recent work on network interface optimization [8, [12] [13] [14] has been focused on simultaneous processing of packets (or messages) from the same or different connections (or flows) rather than on improving the throughput and latency of individual connections. Indeed, early work [7, 15] tried to improve performance using parallelization. Nevertheless, the parallel execution of stateful protocols, such as TCP/IP, shows difficulties related with contention and synchronization costs, and cache efficiency [8, 12, 14] . The coordinated scheduling of protocol processing and network applications on the same cores has been proposed to decrease these overheads. This strategy implies the affinity of interrupts, flows, and network data [14] that means to steer network interrupts, packet processing and received data to the same core where the corresponding network application is been executed. Two main complementary trends for allowing affinity-based network interfaces can be identified: NIC technologies [16] [17] [18] and OS software technologies [8, [12] [13] . While the NIC technologies implement features at the NIC to assign packets to their corresponding flow, distribute them among different receive queues, and steer packets to the cores on which the corresponding network application resides, the OS software technologies try to use a core (or several ones) in the multicore microprocessor to steer the incoming packets to other cores . The work here presented contributes to the approach of OS software technologies as we have devised an approach that distributes the communication tasks of the network interface among the different processors/cores in the node according to their affinity (proximity between the processors/cores where the work is done and the memory where the corresponding data structures are stored), thus trying to improve the performance in latency and bandwidth of the communication path that a given connection uses. Therefore, the existence of heavy loaded cores, that execute not only the network application but also the communication processing, is avoided by distributing these tasks among different cores whilst the locality of data accesses is kept through a suitable affinity scheduling. Although our approach allows the performance improvement of individual connections, it could also be applied to take advantage of either packet-based, or connection-based parallelism in the same way as recent Receive Packet Steering (RPS) and Receive Flow Steering (RFS) technologies.. In what follows, Section 2 describes the characteristics of our affinity-based network interface. Section 3 provides our experimental set up, and the performance evaluation of dynamic web servers and MPI workloads, while different network interface alternatives are implemented. Finally, the previous work on this area is briefly described in Section 4, and the conclusions are given in Section 5. Affinity issues have been considered in this paper for improving network applications performance by using multiple processors/cores. Thus, an affinity-based network interface (AbNI) has been proposed in the paper and its performance has been analyzed in different node configurations, and compared with network interfaces that implement onloading and offloading strategies. The two different node configurations considered (hybrid(1) and hybrid(2)) have been analyzed to take into account the workload data location that could minimize the cache misses, as these configurations present different characteristics on the cache hierarchy. The experimental results obtained by using a simple ping-pong communication, a real dynamic web application, and MPI workloads in a cluster show that the considered AbNI configurations clearly outperform either offloaded or onloaded proposals in terms of throughput and requests per second, and response time has been measured for different computation/communication rates (i.e., application rates) with respect to either offloaded or onloaded proposals. Thus, improvements from 45% to 110% in throughput, from 70% to 500% percent in response time, and from 35% to 85% in requests per second have been obtained depending on the configuration and web server application rates. In most cases the hybrid(2) configuration outperforms hybrid(1) one, as it is expected considering that it provides a better data distribution. The behaviour of the improvements, obtained for different computation/communication profiles in the dynamic web application considered, can be explained by taking into account the decreasing in both communication overheads and computation costs provided by the task distribution and cache layouts of the proposed network interface configurations. On the other hand, experiments using the MPI message passing framework over TCP/IP have been performed. These experiments show improvements for small and large messages. Nevertheless, the improvements using small messages are higher as the application ratio grows due to the increase in the resources available for protocol processing. As in TCP/IP, the hybrid network interfaces provide better MPI communication performances than other offloading and onloading alternatives. When larger messages are transferred (64 KB), the bottleneck on the communication path moves to the internal buses of the node and the benefits of the offloading techniques diminish specially for high values of the parameter γ (ratio of application workload to communication overhead). Nevertheless, the proposed AbNI shows a better behaviour for high values of γ than other approaches and clearly outperforms them in both throughput and latency.. There is still much work to do at the operating system level to improve the communication performance on multicore architectures. We are now considering the effect of our proposed network interface in the context of several connections and, in the future, we will also intend to take into account the effect of cache layouts not considered here, such as the cache hierarchies of three levels, and to evaluate our approaches by using other real network applications. The code of the modified kernel functions and the code (C and python) of the network interface are available upon request to the authors. There have been many researches on network interface optimization to reduce the communication overhead in the host CPUs that run applications. There are many commercial devices, called TCP/IP Offload Engines (TOEs) 11 Ç, [29] , that offload different parts of the TCP/IP protocol stack onto an NIC attached to the I/O bus. Other techniques related with protocol offloading are connection handoff [32] that allows the operating system to control the number of TCP connections offloaded to the NIC, and network interface data caching [33] that reduces traffic along the local buses by caching the frequently-requested data using on-board DRAM included in the programmable NIC. Onloading has been commercially released through implementations such as the Intel I/O acceleration technology (I/OAT) 12 Ç, [11, 34] , which include an onloaded protocol stack as one of its features, along with header splitting, interrupt coalescing, and enhanced DMA transfers through asynchronous I/O copy [23] . There have been some proposals [27, 35] that try to combine the benefits of offloading and onloading to accelerate the network processing by taking advantage of the existence of several processors in the node (not only in the NIC). In [35] , the network stack processing is not fully offloaded to the NIC. Instead, the data processing is done by a specific hardware accelerator while the protocol control operations are implemented by software on a specific CPU as in the onloading strategy. As the two main components of the interface exchange information asynchronously by using two queues, they are loosely coupled and avoid bottlenecks related to their interaction. The interaction between multi-core architectures and 10-Gigabit Ethernet networks is also considered in [36] , where it is demonstrated that adequate mappings of the processes of an application among different cores and the NIC can correct the imbalance created by the protocol stack processing, thus minimizing cache misses and improving the overall performance. In [37] the main physical obstruction to scale the networking performance beyond the gigabits per second is the memory system due to the scalability problem caused by the gap between the improvements on processor speed, link bandwidth, and memory latency and bandwidth. That paper presented a performance study of memory behaviour in a core TCP/IP suite with protocols implemented in the user space because the authors could not access to the source code of the operating system used in the SGI platform where the study was done. It was concluded that 1) in most scenarios instruction cache behaviour becomes decisive for the performance improvement, even for small average packet sizes and zero-copy interfaces, and 2) larger caches and higher associativity improve communication performance (mainly for TCP communication) as many cache misses in packet processing are conflict misses. Moreover, it was also indicated that network protocols should scale with clock speed except for cold caches (cases where caches do not store correct information of data and instructions at the beginning of packet processing) where cache performance shows an important decrease. The parallelization of network protocols and the exploitation of parallelism in network interfaces have also been proposed and analyzed [7] [8] 15, 38] . The different strategies that can be used to parallelize protocols are considered in [7] , where it is shown that, although computer architectural trends can influence the situation, the limited packet-level parallelism achieved in a single TCP connection can be increased in case of multiple connections. In [39] , one or more processors or hardware threads, called packet processing engines (PPEs), are dedicated to processing the network tasks, and asynchronous I/O (AIO) is used to communicate the PPE and the application. The interface between the application and the PPE uses the shared memory through AIO, in order to avoid the operating system overheads associated with socket operations. Thus, a user-level network interface was implemented to bypass the operating system in the network operations and reduce data copies. In this approach, the PPEs substitute softirq by in-line function calls thus avoiding context switches and cache conflicts with the application. The prototype evaluated in [39] by using the open source web server shows small benefits that the authors mainly relate with the use of the AIO and polling, and less with the reduced cache misses. The proposals described here do not try to bypass the operating system but making an efficient use of its characteristics. The TCP/IP stack implemented by the OS kernel is not modified although adequate cache layouts are necessary to optimize the performance. However, many operating system features and optimizations which allow the management of the available resources have been used. This deals with modifications in the softirq Linux scheduling mechanism and the network interface driver to implement a process scheduling scheme which is aware of cache locality to leverage network performance. The effect of affinity in network processing has been considered in [8, 12-14, 35-36, 40-43] , where many researchers have demonstrated that efficient process scheduling and management of multiple cores as well as the cache locality can improve the throughput. These researches reveal that processor affinity and cache locality constitute critical factors for the network performance in multi-core systems. The experiments performed in [40] regarding the advantages of using architectures which take care about the data locality and affinity issues, also have shown that these architectures outperform both offloaded and onloaded network interfaces. Moreover, in applications that use several threads [13, 35] , important improvements can be also achieved by processor affinity. In [13] , a scheduling procedure is proposed to dynamically decide the optimal core affinity of networking processes for multiple network interfaces. It is implemented at the kernel level and takes into account the cache architecture, the communication characteristics, the interrupt affinity, and the core loads to distribute the processes. The proposal shows improvements higher than a 60% in processor utilization and higher than a 30% in the bandwidth and responsiveness performances, by using microbenchmarks and emulating the application-level behaviours. The evaluation of the proposal in real applications such as parallel file systems and web servers is considered in [13] as future work. The paper [42] shows that a multiport Gigabit Ethernet can achieve a bandwidth comparable to 10-Gigabit Ethernet with adequate interrupt affinity configurations. Moreover, from the experimental results obtained in their system, the authors argued that processing packets in many cores could require more resource consumption without benefit due to the locks between the softirqs running in different cores and the low cache efficiency. Goglin [41] presented a multiqueue support for the Open-MX network stack (which bypasses the usual TCP/IP stack), so that the received packets corresponding to the same application process are handled by the same core. The positive effect of this strategy in the cache behaviour and thus in the communication intensive applications was also shown in this paper. Recent studies [14, 16] show different NIC technologies for parallel receive processing in multicore/multiprocessing environments by applying affinity strategies. Direct Cache Access [16] makes it possible to store a received packet in the cache of the core that processes the protocol and executes the network application. Receive side scaling (RSS) 13 Ç supports multiple queues and uses a hash function in the NIC to assign packets of a given flow to the same queue although it cannot steer the received packets to the same core where the corresponding application is running. The Ethernet Flow Director technology 14 Ç introduced by Intel can steer the received packets of a given flow to the same core where the corresponding communication protocol is processed and the extracted data used by the application, although packets reordering is possible. The Transport-Friendly (A-TFN) NIC, recently proposed in [14] , describes an approach to solve this problem.. In the affinity-based network interface here proposed, only one core is mainly involved in the interrupt handler and device driver processing for the specific network application that consumes the packets and the softirq is scheduled on the same core that runs the networking application in order to leverage the cache performance. Moreover, besides MPI loads, we have used dynamic web servers to evaluate our AbNI, by also analyzing their behaviours for different cache layouts and communication and computation loads. We also plan to analyze the behaviour of our proposed network interfaces in the context of multiple NICs, such as it is considered in [13] , by using our approach summarized in Fig.6 . Moreover, recent advances in Linux networking, such as Receive Packet Steering (RPS) and Receive Flow Steering (RFS), have also tried to take advantage of the presence of several cores in the nodes and to avoid the ksoftirqd [19] bottleneck by modifying the softirqs priority to manage packet flooding situations. RPS uses a hash from the protocol head to select the core where the packet is enqueued to be processed (RPS includes the rxhash field to the sk buff structure to take advantage of the capabilities of some NICs that are able to directly calculate the hash value from the received packet). Consequently it is possible that all the packets belonging to the same stream go to the same core thus increasing cache locality. RFS enhances the effects of RPS by applying affinity to steer the incoming packets to the core on which the corresponding application is running. Precisely, we plan to implement these strategies in our distributed interfaces to compare their performances with those provided by our present approaches based on TNAPI.

O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O