NEIGHBOURHOOD GOSSIPING IN HYPERCUBES SATOSHI FUJITA ? Department of Electrical Engineering , Faculty of Engineering , Hiroshima University Kagamiyama 1-4-1 , Higashi-Hiroshima , 739 Japan STEPHANE PERENNESy Laboratoire I3S , 650 Route des Colles , B.P 145 06903 Sophia-Antipolis Cedex , France JOSEPH G. PETERSz School of Computing Science , Simon Fraser University Burnaby , British Columbia , V5A 1S6 , Canada CMPT TR 97-03 School of Computing Science Simon Fraser University January , 1997 ABSTRACT In the neighbourhood gossiping problem , each node of a network starts with a unique message and must learn the messages of all of its neighbours . In this paper , we prove upper and lower bounds for neighbourhood gossiping in hypercubes under the single-port half-duplex and single-port full-duplex communication models . 1 . Introduction In the -LRB- complete -RRB- gossiping problem , each node of a network starts with a unique message and must learn the messages of all other nodes . In the neighbourhood gossiping problem , each node only needs to learn the messages of all of its neighbours . In this paper , we will consider the neighbourhood gossiping problem in hypercubes . We assume a unit-cost single-port model of communication in which a node can communicate with one adjacent node -LRB- neighbour -RRB- at any given time and a communication takes one time unit or round . In the half-duplex model , information can only flow in one direction at a time ; in the full-duplex model , it can flow in both directions simultaneously . Let N = 2n denote the number of nodes in a hypercube with n dimensions , or n-cube . Let g1 -LRB- n -RRB- denote the time for -LRB- complete -RRB- gossiping in an n-cube under the single-port half-duplex communication model . Similarly , let g2 -LRB- n -RRB- denote the time ? Research partially supported by the Ministry of Education , Science and Culture of Japan -LRB- Grant # 08680372 -RRB- yResearch supported by the European project MAP zResearch supported by NSERC of Canada and CNRS - France for the single-port full-duplex model . It is obvious that g2 -LRB- n -RRB- = n . The best current bounds for g1 -LRB- n -RRB- are 1:44 n + O -LRB- 1 -RRB- < = g1 -LRB- n -RRB- -LSB- 1,2 -RSB- and g1 -LRB- n -RRB- < = 1:88 n + O -LRB- 1 -RRB- -LSB- 3 -RSB- . The following bounds for neighbourhood gossiping are immediate . Observation 1 Neighbourhood gossiping in an n-cube requires at least dlog2 ne rounds , at most 2n rounds in the half-duplex model , and at most n rounds in the full-duplex model . The upper bounds are due to \ naive '' algorithms which simply exchange information between all pairs of neighbouring nodes using two rounds for each dimension in the half-duplex model and one round for each dimension in the full duplex model . Of course , the complete gossiping algorithm in -LSB- 3 -RSB- gives a better upper bound of 1:88 n + O -LRB- 1 -RRB- for the half-duplex model . The lower bound -LRB- for both models -RRB- follows from the lower bound for a node to broadcast a message to n other nodes under the single-port model . In this paper , we will establish stronger bounds for both the half-duplex and full-duplex single-port unit-cost communication models . In Section 2 we present neighbourhood gossiping algorithms that are exponentially faster than the best complete gossiping algorithms . In Section 3 , we use arguments similar to the arguments in -LSB- 1,2 -RSB- to improve the lower bounds . Our new upper and lower bounds show that the number of rounds h1 -LRB- n -RRB- to neighbourhood gossip in the n-cube under the half-duplex model is 2:88 log2 n + O -LRB- 1 -RRB- < = h1 -LRB- n -RRB- < = 3:76 log2 n + O -LRB- 1 -RRB- and h2 -LRB- n -RRB- = 2 log2 n + O -LRB- 1 -RRB- under the full-duplex model . A related problem , neighbourhood broadcasting in hypercubes , in which a single originating node must broadcast its information to all of its neighbours , was considered in -LSB- 4 -RSB- . The O -LRB- dlog2 ne -RRB- neighbourhood broadcasting algorithm in -LSB- 4 -RSB- is asymptotically optimal for both the half-duplex and full-duplex models . See -LSB- 5 -RSB- for a discussion of models and results for broadcasting and gossiping with unit-cost models and -LSB- 6,7 -RSB- for comprehensive surveys . 2 . Upper Bounds Theorem 1 The number of rounds to finish neighbourhood gossiping in an ncube is at most 2g1 -LRB- dlog2 ne -RRB- + 2 under the single-port half-duplex communication model , and at most 2dlog2 ne + 2 under the single-port full-duplex model . Proof . Let m = dlog2 ne . Partition V into 2n ? m m-cubes , in such a way that each m-cube consists of all nodes with the same prefix of length n ? m. Let V0 ; V1 ; : : : ; V2n?m ? 1 be the resulting m-cubes , where the subscript is the decimal representation of the associated prefix of length n ? m . The neighbourhood gossiping algorithm consists of three phases . In the first and third phases , each m-cube completes a -LRB- normal -RRB- gossip independently and in parallel . Clearly , each of these two phases takes g1 -LRB- m -RRB- = g1 -LRB- dlog2 ne -RRB- rounds in the half-duplex model and g2 -LRB- dlog2 ne -RRB- rounds in the full-duplex model . In the second phase , messages collected in the first phase are exchanged among neighbouring m-cubes in two rounds as follows . Partition the set of m-cubes into two groups according to the parity of the prefix of length n ? m. For example , V0 , V3 , and V5 are contained in the group of even parity and V1 , V2 , and V4 are contained in the group of odd parity . Let Veven denote the group of m-cubes of even parity and Vodd the group of m-cubes of odd parity . In each m-cube in Veven , any node whose suffix of length m has even parity acts as a sender of the m-cube and any node whose suffix of length m has odd parity acts as a receiver of the m-cube . On the other hand , in each m-cube in Vodd , the nodes whose suffixes of length m have odd parity act as senders of the m-cube and the nodes whose suffixes of length m have even parity act as receivers of the m-cube . Note that if a node in the n-cube is a sender , then its neighbours with respect to the prefix of length n ? m are all receivers , and vice versa . Furthermore , each m-cube contains 2m ? 1 senders and 2m ? 1 receivers . Since m = dlog2 ne , 2m ? 1 > = n = 2 . To exchange messages among neighbouring m-cubes , we partition the set of n ? m dimensions in the prefix of length n ? m into d -LRB- n ? m -RRB- = 2e subsets of size at most two each . We then assign each subset to nodes in V in such a way that nodes with the same suffix of length m are assigned the same subset . This guarantees that the 2m ? 1 sender-receiver pairs cover all n ? m dimensions . Since there are no conflicts , it requires at most two rounds to finish the second phase . To establish the correctness of the algorithm , we note that all neighbours of any node are in the node 's own m-cube or neighbouring m-cubes and that each m-cube receives at least one message from each neighbouring m-cube during the second phase . 2 Corollary 1 Neighbourhood gossiping in the n-cube takes at most 3:76 dlog2 ne + O -LRB- 1 -RRB- rounds under the single-port half-duplex communication model . The bounds for the half-duplex model in Theorem 1 and Corollary 1 are based on the asymptotic result in -LSB- 3 -RSB- . For small n , we can use a more straightforward gossiping algorithm for Phases 1 and 3 to establish non-asymptotic bounds . The usual -LRB- complete -RRB- gossiping algorithm for m-cubes uses two rounds for each dimension . Using this algorithm in Phases 1 and 3 , and two rounds in the second phase gives an upper bound of 4dlog2 ne + 2 . We can eliminate one round from each of Phases 1 and 3 by noting that complete gossiping is not necessary . The objective of Phase 1 is to collect all messages in an m-cube into all of its senders and the objective 00 01 10 11 00 01 10 11 00 01 10 11 Phase 1 senders 00 01 10 11 00 01 10 11 00 01 10 11 Phase 3 receivers Fig. 1 . Phases 1 and 3 for m = 2 . of Phase 3 is to disseminate the messages received from neighbouring m-cubes to all other nodes in the m-cube . If m = 1 , it takes only one round for each of these phases . As can be seen in Fig. 1 , Phases 1 and 3 take only three rounds each when m = 2 . If m = 3 , it takes five rounds as shown in Fig. 2 . In general , Phases 1 and 3 can be completed in 2m ? 1 rounds each for any m. Step 3 Step 4 Step 5Step 2Step 1 Step 3 Step 4 Step 5Step 2Step 1 -LRB- b -RRB- Phase 3 . senders -LRB- a -RRB- Phase 1 . receivers Fig. 2 . Phases 1 and 3 for m = 3 . Corollary 2 It takes at most 4dlog2 ne rounds to finish neighbourhood gossiping in the n-cube under the single-port half-duplex communication model . Table 1 shows the best gossiping times that we have obtained for small n. Phases 1 and 3 use the method of Corollary 2 . Phase 2 uses ad hoc methods . In the table , the values of m were chosen to minimize the total gossiping time in combination with Phase 2 . 3 . Lower Bounds The demonstration of the lower bounds relies on two arguments . The first concerns concurrent broadcasting and gathering . The second uses the fact that the set of edges in any fixed dimension in a hypercube separates the graph into two parts . In the full-duplex model , one node can inform or receive the information from as many as 2t nodes -LRB- including itself -RRB- in t steps and either operation can be performed Table 1 . Upper bounds on half-duplex neighbourhood gossiping time . n m Gossiping time dlog2 ne 3 1 4 -LRB- = 1 + 2 + 1 -RRB- 2 4 1 5 -LRB- = 1 + 3 + 1 -RRB- 2 5 1 6 -LRB- = 1 + 4 + 1 -RRB- 3 6 1 7 -LRB- = 1 + 5 + 1 -RRB- 3 7 1 8 -LRB- = 1 + 6 + 1 -RRB- 3 8 1 9 -LRB- = 1 + 7 + 1 -RRB- 3 9 1 10 -LRB- = 1 + 8 + 1 -RRB- 4 9 ; 10 2 10 -LRB- = 3 + 4 + 3 -RRB- 4 11 ; 12 2 11 -LRB- = 3 + 5 + 3 -RRB- 4 13 ; 14 2 12 -LRB- = 3 + 6 + 3 -RRB- 4 15 ; 16 2 13 -LRB- = 3 + 7 + 3 -RRB- 4 17 ; 18 2 14 -LRB- = 3 + 8 + 3 -RRB- 5 19 ; 20 2 15 -LRB- = 3 + 9 + 3 -RRB- 5 20 ? 23 3 15 -LRB- = 5 + 5 + 5 -RRB- 5 24 ? 27 3 16 -LRB- = 5 + 6 + 5 -RRB- 5 28 ? 31 3 17 -LRB- = 5 + 7 + 5 -RRB- 5 32 ? 35 3 18 -LRB- = 5 + 8 + 5 -RRB- 5 29 ? 36 4 18 -LRB- = 7 + 4 + 7 -RRB- 5 ; 6 37 ? 44 4 19 -LRB- = 7 + 5 + 7 -RRB- 6 45 ? 52 4 20 -LRB- = 7 + 6 + 7 -RRB- 6 53 ? 60 4 21 -LRB- = 7 + 7 + 7 -RRB- 6 61 ? 68 4 22 -LRB- = 7 + 8 + 7 -RRB- 6 ; 7 54 ? 69 5 22 -LRB- = 9 + 4 + 9 -RRB- 7 70 ? 85 5 23 -LRB- = 9 + 5 + 9 -RRB- 7 concurrently by all of the nodes . This result is not true under the half-duplex model . The next lemma establishes upper bounds on the rates that nodes can distribute and receive information . The argument is similar to arguments for the gossiping problem in -LSB- 1,2 -RSB- . We refer the reader to -LSB- 1,2 -RSB- for more detailed developments of the arguments than we present here . Definition 1 Given a network G with N nodes v1 ; v2 ; : : : ; vN , and a communication protocol in G , Ii -LRB- t -RRB- is the number of nodes that have been informed by node vi -LRB- including itself -RRB- during the first t rounds , and Ri -LRB- t -RRB- is the number of messages that vi has received -LRB- including its own -RRB- by round t. Let I -LRB- t -RRB- = P vi Ii -LRB- t -RRB- and R -LRB- t -RRB- = P vi Ri -LRB- t -RRB- . Lemma 1 In a network G with N nodes , I -LRB- t -RRB- < = N aet , and R -LRB- t -RRB- < = N aet , where ae = 2 for the full-duplex model and ae = 1 + p5 2 for half-duplex model . Proof . Let M -LRB- t -RRB- be the N ? N matrix with entries Mij -LRB- t -RRB- 2 f0 ; 1g that describes the communications that take place in round t : Mii -LRB- t -RRB- = 1 and for i < > j , Mij -LRB- t -RRB- = 1 iff node vi sends a message to node vj at time t. Let M -LRB- t -RRB- denote the matrix of communications completed during the first t rounds . The -LRB- i ; j -RRB- entry of M -LRB- t -RRB- is 1 if the message of node vi reaches node vj within the first t rounds and the entry is otherwise . Note that M -LRB- t -RRB- is a product of communication matrices : M -LRB- t -RRB- = M -LRB- t ? 1 -RRB- ? M -LRB- t -RRB- and M -LRB- 0 -RRB- = I -LRB- the identity matrix -RRB- . Also note that 0-1 matrix multiplication is being used here . Let I -LRB- t -RRB- denote the column vector -LRB- I1 -LRB- t -RRB- ; I2 -LRB- t -RRB- ; : : : ; IN -LRB- t -RRB- -RRB- T , let 1 be the vector of length N with all entries equal to 1 , and let the column vector 1T be the transpose of 1 . Then I -LRB- t -RRB- = M -LRB- t -RRB- ? 1T and I -LRB- t -RRB- = P vi Ii -LRB- t -RRB- = 1 ? I -LRB- t -RRB- = 1 ? M -LRB- t -RRB- ? 1T . Similarly , R -LRB- t -RRB- = 1 ? MT -LRB- t -RRB- ? 1T = I -LRB- t -RRB- . Let jj : : jj denote the Euclidean norm for vectors and the associated spectral norm for matrices -LRB- see -LSB- 8 -RSB- for definitions and properties -RRB- . It is shown in -LSB- 1,2 -RSB- -RRB- that if M is a communication matrix describing a round of a communication protocol , then jjM jj < = ae , with ae = 2 for the full-duplex model and ae = 1 + p5 2 for the half-duplex model . It follows that , at time t , jjI -LRB- t -RRB- jj = jjM -LRB- t -RRB- ? 1T jj < = aet ? jj1T jj = aetp N : -LRB- 1 -RRB- The value jjI -LRB- t -RRB- jj is minimized when all the coordinates of I -LRB- t -RRB- are equal -LSB- 1,2 -RSB- , so jjI -LRB- t -RRB- jj = jj -LRB- I1 -LRB- t -RRB- ; I2 -LRB- t -RRB- ; : : : ; IN -LRB- t -RRB- -RRB- T jj > = ? I -LRB- t -RRB- N ; I -LRB- t -RRB- N ; : : : ; I -LRB- t -RRB- N ? T = I -LRB- t -RRB- pN : -LRB- 2 -RRB- Combining Eq . -LRB- 1 -RRB- and Eq . -LRB- 2 -RRB- we get I -LRB- t -RRB- < = aetN . The same argument can be used to show that R -LRB- t -RRB- < = aetN by renumbering the rounds in reverse order , exchanging senders and receivers , and transposing the communication matrices . 2 We need one more lemma before we prove the lower bounds . Definition 2 A message which is originated by a node v in an n-cube and which reaches the dimension i neighbour of v before the end of the communication protocol is valid at time t if the message first uses a dimension i edge in round t . The number of valid messages at a given time t will be denoted f low -LRB- t -RRB- . Lemma 2 The number of valid messages during a neighbourhood gossiping algorithm in the n-cube is at least n2n . That is , P t f low -LRB- t -RRB- > = n2n . Proof . In neighbourhood gossiping , each node must send its message to a neighbour in each dimension . Since the removal of all edges in a dimension i separates the n-cube into two parts , the number of valid messages crossing dimension i during a neighbourhood gossiping protocol is N = 2n . Summing over all n dimensions gives the result . 2 Theorem 2 The number of rounds to complete neighbourhood gossiping in the n-cube is at least 2 logae -LRB- n -RRB- + O -LRB- 1 -RRB- where ae = 2 for the full-duplex model and ae = 1 + p5 2 for the half-duplex model . Proof . Let T be the number of rounds in a neighbourhood gossiping protocol . First , we point out that if two or more messages which are valid at time t are concatenated and sent during the same call on an edge in dimension i then they must have different originators , and since each originator has a different neighbour in dimension i , the messages must have different destinations . Thus , at most one of the originators of the concatenated messages can be informing its dimension i neighbour during round t . The rest of the concatenated messages must have been forwarded at earlier times . So , summing over all nodes gives f low -LRB- t -RRB- < = P vi Ri -LRB- t -RRB- . Similarly , at most one of the concatenated messages can reach its final destination during round t . The rest of the concatenated messages will have to be forwarded during the last T?t rounds , so f low -LRB- t -RRB- < = P vi Ii -LRB- T ? t + 1 -RRB- . Putting these two inequalities together gives f low -LRB- t -RRB- < = min -LRB- P vi Ii -LRB- T ? t + 1 -RRB- ; P vi Ri -LRB- t -RRB- -RRB- = min -LRB- I -LRB- T ? t +1 -RRB- ; R -LRB- t -RRB- -RRB- . From Lemma 1 , we have I -LRB- T ? t + 1 -RRB- < = N aeT ? t +1 and R -LRB- t -RRB- < = N aet . For t < = T2 ; f low -LRB- t -RRB- < = R -LRB- t -RRB- < = N aet . For t > T2 ; f low -LRB- t -RRB- < = I -LRB- T ? t + 1 -RRB- < = N aeT ? t +1 . The sum of these two exponential series is T X t = 1 f low -LRB- t -RRB- < = bT = 2c X t = 1 N aet + T X t = bT = 2c +1 N aeT ? t +1 < = 2N aedT = 2e +2 : Since PTt = 1 f low -LRB- t -RRB- > = nN by Lemma 2 , we have T > = 2logae -LRB- n -RRB- + O -LRB- 1 -RRB- . 2 Corollary 3 Neighbourhood gossiping in the n-cube takes at least 2:88 log2 n + O -LRB- 1 -RRB- rounds under the single-port half-duplex communication model and at least 2 log2 n + O -LRB- 1 -RRB- rounds under the single-port full-duplex model . References -LSB- 1 -RSB- S. Even and B. Monien , On the number of rounds necessary to disseminate information , in Proc . 1st ACM Symposium on Parallel Algorithms and Architectures , Santa Fe , NM , June 1989 , 318 -LCB- 327 . -LSB- 2 -RSB- D.W. Krumme , G. Cybenko , and K.N. Venkataraman , Gossiping in minimal time , SIAM J. Comput . 21 -LRB- 1992 -RRB- 111 -LCB- 139 . -LSB- 3 -RSB- D.W. Krumme , Fast gossiping for the hypercube , SIAM J. Comput . 21 -LRB- 1992 -RRB- 365 -LCB- 380 . -LSB- 4 -RSB- J.-C . Bermond , A. Ferreira , and J.G. Peters , Partial broadcasting in hypercubes , Int . Workshop on Interconnection Networks , Luminy , France , July 1991 . -LSB- 5 -RSB- J. Hromovi < = c , R. Klasing , B. Monien , and R. Peine , Dissemination of information in interconnection networks -LRB- broadcasting and gossiping -RRB- , in Combinatorial Network Theory , eds . D.-Z . Zhu and D.F. Hsu -LRB- Kluwer Academic Publishers , 1995 -RRB- 125 -LCB- 212 . -LSB- 6 -RSB- S.M. Hedetniemi , S.T. Hedetniemi , and A.L. Liestman , A survey of gossiping and broadcasting in communication networks , Networks 18 , 1986 , 319 -LCB- 349 . -LSB- 7 -RSB- P. Fraigniaud and E. Lazard , Methods and problems of communication in usual networks , Disc . Appl . Math . 53 -LRB- 1994 -RRB- 79 -LCB- 133 . -LSB- 8 -RSB- G.H. Golub and C. Van Loan , Matrix Computations -LRB- The Johns Hopkins University Press , Baltimore , 1984 -RRB- .

O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O