An Adaptive Generic Sorting Algorithm that Uses Variable Partitioningz Vladimir Estivill-Castro ? and Derick Woody Technical Report HKUST-CS96-19 June 1996 ? School of Information Systems Queensland University of Technology Red Hill Q4059 Australia yDepartment of Computer Science Hong Kong University of Science & Technology Clear Water Bay , Kowloon Hong Kong Abstract A sorting algorithm is adaptive if its run time , for inputs of the same size n , varies smoothly from O -LRB- n -RRB- to O -LRB- n log n -RRB- as the disorder of the input varies . Recently , researchers have focused their attention on sorting algorithms that are optimally adaptive with respect to several measures of disorder , since the type of disorder in the input is usually not known . They have demonstrated a need to develop tools for the construction of sorting algorithms that are adaptive with respect to large classes of measures . We present Partition Sort , a new generic sorting algorithm for which we can establish adaptivity . It is based on divide-and-conquer in which the number of subproblems at each level of recursion depends on the disorder of the input . We apply this generic algorithm to obtain optimal adaptivity for several measures of disorder . In addition , we introduce Randomized Partition Sort , a randomized version of Partition Sort , that not only is adaptive in the expected case , but also simplifies implementations . zThis research was supported by grants from the Natural Sciences and Engineering Research Council of Canada , from the Information Technology Research Centre of Ontario , and from the Research Grants Committee of Hong Kong . The Hong Kong University of Science & Technology Technical Report Series Department of Computer Science An Adaptive Generic Sorting Algorithm that Uses Variable Partitioning1 Vladimir Estivill-Castro2 Derick Wood3 Abstract A sorting algorithm is adaptive if its run time , for inputs of the same size n , varies smoothly from O -LRB- n -RRB- to O -LRB- n log n -RRB- as the disorder of the input varies . It is well accepted that files that are already sorted are often sorted again and that nearly sorted files often occur . Recently , researchers have focused their attention on sorting algorithms that are optimally adaptive with respect to several measures of disorder , since the type of disorder in the input is usually not known . They have demonstrated a need to develop tools for the construction of sorting algorithms that are adaptive with respect to large classes of measures . We present Partition Sort , a new generic sorting algorithm for which we can establish adaptivity . It is based on divide-and-conquer in which the number of subproblems at each level of recursion depends on the disorder of the input . We apply this generic algorithm to obtain optimal adaptivity for several measures of disorder . In addition , we introduce Randomized Partition Sort , a randomized version of Partition Sort , that not only is adaptive in the expected case , but also simplifies implementations -LRB- for example , it avoids median finding -RRB- . 1 Introduction Intuitively , a sorting algorithm is adaptive if its run time , for inputs of the same size n , varies smoothly from O -LRB- n -RRB- to O -LRB- n log n -RRB- as the disorder or nearly-sortedness of the input varies and it adapts its performance without knowing the disorder in advance . It is well accepted that sorted files are often sorted again and that nearly sorted files often occur . Thus , adaptive sorting is not only theoretically interesting but also it promises to improve current practice -LSB- 4 -RSB- . One critical issue that we face when we design adaptive sorting algorithms is how to measure disorder or nearly-sortedness . There are a number of measures of disorder -LSB- 5 -RSB- ; the most well-known measure is the number Inv -LRB- X -RRB- of inversions in a sequence X = hx1 ; : : : ; xni , where an inversion is a pair -LRB- i ; j -RRB- of positions in X such that i < j and xi > xj . It is certainly useful to design a sorting algorithm that is adaptive for a specific measure ; however , the diversity of measures of disorder suggests that an algorithm that is adaptive for several measures is much more useful , since in the general setting , the type of disorder in the input is usually not known . Initially , most developments arose from the study of specific measures , but increasingly , researchers have focused their attention on sorting algorithms that are adaptive for several measures -LSB- 5 -RSB- . Unfortunately , experience has shown that , whenever we obtain a sorting algorithm that is adaptive for more measures than previous algorithms , its implementation is more complex -LRB- for example , more complex data structures to represent the sorted sequence in Insertion Sort can make the algorithm adaptive 1This work was partially carried out under grants from the Natural Sciences and Engineering Research Council of Canada and the Information Technology Research Centre of Ontario . 2School of Information Systems , Queensland University of Technology , Red Hill Q4059 , Australia . E-mail : vlad@icis.qut.edu.au . 3Department of Computer Science , Hong Kong University of Science and Technology , Clear Water Bay , Kowloon , Hong Kong . E-mail : dwood@cs.ust.hk . 2 Estivill-Castro and Wood for more measures -LSB- 5 -RSB- -RRB- ; thus , the new algorithm is impractical . There is a need to develop tools for the construction of practical sorting algorithms that are adaptive for large classes of measures of disorder . Divide-and-conquer is a general and extremely successful strategy for the design and analysis of algorithms ; in particular , it is the basis of many sorting algorithms for the comparison-based model of computation . Recently , Estivill-Castro and Wood -LSB- 4 -RSB- studied divide-and-conquer as the basis of a universal method for the development of adaptive sorting algorithms . The design of generic sorting algorithms has at least two advantages : ffl Algorithm designers can focus their efforts on the combinatorial properties of the measures of disorder that are of interest , rather than in the combinatorial properties of the algorithm . ffl Algorithm designers can regulate the trade-off between the number of measures for which the sorting algorithm is adaptive and the complexity of its implementation . A generic adaptive sorting algorithm that we designed previously -LSB- 4 -RSB- requires that the division protocol partition the data into a fixed number of parts . Now , we present new generic adaptive sorting algorithm that uses variable partitioning . Not only does this new algorithm have the preceding advantages but also , because of the added flexibility , it enables us to obtain optimal adaptivity for even more measures . In Section 2 , we introduce some measures of disorder . In Section 3 , we present Partition Sort , the new generic adaptive sorting algorithm and we present four applications of Partition Sort in Section 4 . In Section 5 , we present its randomized version and , in Section 6 , we present an application in which we eliminate median finding . We conclude with some final remarks in Section 7 . In the following , we denote the base 2 logarithm with log and the set of all finite sequences of integers with N < N . We denote the length of a sequence X with jX j and the cardinality of a set S with kSk . 2 Evaluating disorder To express the performance of a sorting algorithm in terms of the input length and disorder , we must assess the disorder in the input . Intuitively , a measure of disorder is a function that is minimized when a sequence has no disorder and depends on only the relative order of the elements in the sequence . We describe the most common measures of disorder , many others can be found in the literature -LSB- 5 -RSB- . We may consider that , in terms of the disorder it represents , an inversion pair , whose elements are far apart , is more significant than an inversion pair , whose elements are closer . We define Dis as the largest distance determined by an inversion . Often local disorder is not as important as global disorder ; we define Max as the largest distance an element must travel to reach its sorted position . Ascending runs constitute sorted segments of the data . We define Runs -LRB- X -RRB- as the minimum number of contiguous up-sequences that are required to cover X . A natural generalization of Runs is the minimum number SUS -LRB- X -RRB- of ascending subsequences that are required to cover X . We can generalize this measure again and define SMS -LRB- X -RRB- -LRB- for shuffled monotone subsequence -RRB- as the minimum number of monotone -LRB- ascending or descending -RRB- subsequences that are required to cover X . For example , for W0 = h6 ; 5 ; 8 ; 7 ; 10 ; 9 ; 4 ; 3 ; 2 ; 1i , Runs -LRB- W0 -RRB- = 8 , SUS -LRB- W0 -RRB- = kfh6 ; 8 ; 10i ; h5 ; 7 ; 9i ; h4i ; h3i ; h2i ; h1igk = 6 , and SMS -LRB- X -RRB- = kfh6 ; 8 ; 10i ; h5 ; 7 ; 9i ; h4 ; 3 ; 2 ; 1igk = 3 . These examples show that disorder can be evaluated in many ways and different measures take account of different types of disorder . We wish to design sorting algorithms that are optimally adaptive to as many measures as possible . A sorting algorithm is optimally adaptive with respect to a measure M if it achieves , under the comparison-based model of computation , maximal adaptivity with respect to M -LSB- 11 -RSB- . Thus , we concentrate on comparison-based sorting algorithms in which the numbers of data moves and other operations -LRB- for example , pointer assignments -RRB- are bounded by the number of comparisons . Also , when we say that a sorting algorithm is optimally adaptive An Adaptive Generic Sorting Algorithm that Uses Variable Partitioning 3 with respect to a measure we implicitly assume sequences of distinct elements -LSB- 11 -RSB- . We can handle sequences with repeated elements in at least two ways . One way is to extend the measures of disorder ; for example , we can define SUS to be the minimum number of nondecreasing sequences . Alternatively , if stability is important , then we can tag the elements with their initial position to obtain distinct composite keys . The performance bounds that we obtain hold for both points of view and they illustrate that the algorithms are also adaptive to the appearance of multiple keys in the input . 3 Partition Sort The structure of the generic sorting algorithm , Partition Sort , should not be surprising : It uses divide and conquer . What is novel , is that we establish adaptivity with respect to a measure M of disorder by ensuring that the partition protocol satisfies certain requirements . Partition Sort divides the input into s parts , in which the number s of parts is not necessarily fixed and depends on the disorder in the input . If we can not divide a sequence into smaller sequences or it is so close to sorted order that we can sort it by an alternative sorting algorithm , then we consider it to be simple . The sizes of the individual parts are not restricted , but the total size of the simple parts must be at least a fixed fraction of the total size of the parts . Sophisticated partition protocols that may require more than linear time may be used . Sorting can be considered a process in which the disorder in the input is reduced to zero . If the partition protocol repeatedly introduces disorder , then we are producing smaller and harder instances of sorting problems and we do not achieve adaptivity . Thus , the partition protocol that we use must ensure that the disorder has been substantially reduced in a fixed fraction of the parts . We formalize this restriction later . For a sequence X , we say that a set P -LRB- X -RRB- of nonempty sequences X1 ; : : : ; Xp is a partition of X if every element of X is in one and only one sequence in P -LRB- X -RRB- and the sequences in P -LRB- X -RRB- have elements only from X . The value p is called the size of the partition . Partition Sort works as follows . Given a sequence X , a partition protocol computes a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg . Note that the size of the partition depends on the input X . Each simple Xi is sorted using a secondary sorting algorithm that sorts simple sequences . Each nonsimple part is sorted by a recursive call of Partition Sort . In the final step , all the sorted parts are merged ; see Figure 1 . We can merge kP -LRB- X -RRB- k sorted parts in O -LRB- jX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- -RRB- time -LRB- pairing the parts X2i ? 1 and X2i ; for i = 1 ; : : : ; bkP -LRB- X -RRB- k = 2c , and merging them reduces the number of parts by a half in linear time -RRB- . If Partition Sort takes no more time to compute the partition and resolve its recursive calls , then we obtain a sorting algorithm that takes O -LRB- jX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- -RRB- total time . If kP -LRB- X -RRB- k is related to a measure M of disorder , then Partition Sort is adaptive with respect to M . We can ensure that Partition Sort resolves its recursive calls with the claimed worst-case performance only if a constant fraction of its recursive calls are simple and the sizes of the partitions obtained during further recursive calls do not increase arbitrarily . We make these notions precise in the following theorem . Theorem 3.1 Let c ; d ; k ; 2 < be constants with < c < = 1 , d > and k > . Let PP be a partition protocol such that , for all sequences X 2 N < N : 1 . To detect that X is nonsimple , to create a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg of X using PP , and to merge the sorted versions of X1 ; : : : ; XkP -LRB- X -RRB- k takes no more than djX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons ; 2 . The sum of the lengths of the simple sequences in P -LRB- X -RRB- is at least cjX j ; 3 . For all Xi in P -LRB- X -RRB- , if PP is applied to Xi , then no more than kP -LRB- X -RRB- k sequences are obtained ; 4 . There is a sorting algorithm that sorts any simple Xi making no more than kjXij log -LRB- kP -LRB- X -RRB- k + 1 -RRB- comparisons . 4 Estivill-Castro and Wood Partition Sort -LRB- X -RRB- X is sorted . Terminate . X is simple . Sort it using an algorithm for simple sequences . X is neither sorted nor simple . Create a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg For i = 1 ; : : : ; kP -LRB- X -RRB- k , sort Xi recursively Merge the sorted sequences to give X in sorted order Figure 1 : Partition Sort . The generic sorting algorithm . Then , Partition Sort makes O -LRB- jX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- kj + 1 -RSB- -RRB- -RRB- comparisons . In other words , Partition Sort is adaptive with respect to the size of the initial partition . Sketch of the proof of Theorem 3.1 : Since PkP -LRB- X -RRB- k i > = 1 & Xiis simple jXij > = cjX j ; for all nonsimple sequences X , there is a constant u with u > = d + k PkP -LRB- X -RRB- k -LRB- i > = 1 -RRB- & Xiis simple jXij = jX j 1 ? PkP -LRB- X -RRB- k -LRB- i > = 1 -RRB- & Xinonsimple jXij = jX j : -LRB- 1 -RRB- Now it is straightforward to prove , by induction on the length of X , that , for all sequences X 2 N < N , Partition Sort makes no more than ujX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons . We present only the induction step since the basis is immediate . If X is simple , then X is sorted by an alternative algorithm in the required bound . If X is nonsimple , then Partition Sort makes at most djX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k +1 -RSB- -RRB- comparisons to detect that X is nonsimple , to construct the partition , and to merge the results of the recursive calls . We now account for the time required to resolve the recursive calls . For all simple Xi in the partition , we use no more than k PkP -LRB- X -RRB- k -LRB- i > = 1 -RRB- & Xiis simple jXij -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons in total . For all nonsimple Xi in the partition , by the induction hypothesis , we use no more than PkP -LRB- X -RRB- k -LRB- i > = 1 -RRB- & Xiis nonsimple ujXij -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons in total . Thus , Partition Sort requires no more than djX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- + k kP -LRB- X -RRB- k X -LRB- i > = 1 -RRB- & Xiis simple jXij -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- + u kP -LRB- X -RRB- k X -LRB- i > = 1 -RRB- & Xiis nonsimple jXij -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons in total and , by Equation 1 , it takes no more than ujX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- comparisons . 2 Before we present some applications of Partition Sort , we make two observations . First , Generic Sort -LSB- 4 -RSB- can be regarded as a variant of Partition Sort ; the specific form of Generic Sort , however , results in a theorem that is easier to apply . Second , Chen and Carlsson -LSB- 2 -RSB- defined generic measures of disorder as \ : : : minimal size of a specific type o of partition ; '' however , they did not use this notion with the generality given by Theorem 3.1 . 4 Applications of Partition Sort We first apply Partition Sort to Natural Mergesort -LSB- 9 , page 161 -RSB- , which partitions the input into ascending runs . A sequence is simple if it is sorted and , in fact , in this application all parts of the An Adaptive Generic Sorting Algorithm that Uses Variable Partitioning 5 partition are simple . We obtain a partition into ascending runs , in linear time , by scanning the input once . Since the number of ascending runs is directly related to the measure Runs , see Section 2 , Natural Mergesort takes O -LRB- jX j -LRB- 1 + log -LSB- Runs -LRB- X -RRB- + 1 -RSB- -RRB- -RRB- time and is , therefore , Runs-optimal ; a new proof of Mannila 's result -LSB- 11 -RSB- . Next , we use Partition Sort to obtain a sorting algorithm that is optimally adaptive with respect to Dis . For a given sequence X , the partition protocol computes , in O -LRB- jX j -RRB- time , the value Dis -LRB- X -RRB- as described , by Altman and Igarashi -LSB- 1 -RSB- . Then , it divides X into Dis -LRB- X -RRB- + 1 subsequences , each containing the elements that are Dis -LRB- X -RRB- + 1 elements apart -LRB- for example , if Dis -LRB- X -RRB- is one , then the partition protocol gives two subsequences , one of elements in even positions and one of elements in odd positions -RRB- . By definition , after partitioning , all parts are sorted and , therefore , simple . By Theorem 3.1 , the algorithm sorts X in O -LRB- jX j -LRB- 1 + log -LSB- Dis -LRB- X -RRB- +1 -RSB- -RRB- time , which is optimally adaptive with respect to Dis and Max -LSB- 3 -RSB- . Our third application is to Skiena 's Melsort , which we now describe -LSB- 13 -RSB- . When Melsort is applied to an input sequence X , it constructs a partition of the input that consists of a set of sorted lists called the encroaching lists of X . Since encroaching lists are sorted , the simple parts are sorted sequences and , after the partition , all parts are simple . In the final step , Melsort merges the lists to obtain the elements of X in sorted order . The encroaching set of a sequence X = hx1 ; : : : ; xni is defined by the following procedure : We say that xi fits a double-ended queue D if xi can be added to either the beginning or the end of D to maintain D in sorted order . We start with x1 as the only element in the first double-ended queue and , for i = 2 ; : : : ; jX j , we insert xi into the first double-ended queue that it fits . We create a new double-ended queue if xi does not fit any existing double-ended queue . An example should make this process clear . Consider the sequence W3 = h4 ; 6 ; 5 ; 2 ; 9 ; 1 ; 3 ; 8 ; ; 7i : Initially , D1 consists of 4 . The second element , 6 , fits at the end of D1 . The third element , 5 , is between 4 and 6 , so 5 is added to an empty D2 . The next three elements all fit D1 and are placed there . The element 3 does not fit D1 ; but it fits D2 . Similarly , 8 fits D2 and fits D1 , but the last element requires a new double-ended queue . The final encroaching set is fD1 = -LSB- 0 ; 1 ; 2 ; 4 ; 6 ; 9 -RSB- ; D2 = -LSB- 3 ; 5 ; 8 -RSB- ; D3 = -LSB- 7 -RSB- g : The number of lists in the encroaching set is the measure Enc of disorder ; thus , in the example , Enc -LRB- h4 ; 6 ; 5 ; 2 ; 9 ; 1 ; 3 ; 8 ; ; 7i -RRB- = 3 : Since the encroaching set can be constructed in O -LRB- jX j -LRB- 1 + log -LSB- Enc -LRB- X -RRB- + 1 -RSB- -RRB- -RRB- time , Melsort takes O -LRB- jX j -LRB- 1 + log -LSB- Enc -LRB- X -RRB- + 1 -RSB- -RRB- -RRB- time ; therefore , Melsort is Enc-optimal and we have obtained a new proof of Skiena 's result -LSB- 13 -RSB- . Our fourth application is to Slab Sort -LSB- 10 -RSB- , a sorting algorithm that achieves optimality with respect to SMS -LRB- X -RRB- -LRB- the minimum number of shuffled monotone subsequences of X -RRB- and , therefore , optimality with respect to Dis , Max , Runs and SUS . Although Slab Sort is an important theoretical breakthrough it has limited practical value because it requires repeated median finding . Before describing Slab Sort , we must define zig-zag shuffles because , in this application , zig-zag shuffles are the simple sequences . Definition 4.1 Letting X 2 N < N , we denote a subsequence of X by hxi -LRB- 1 -RRB- ; : : : ; xi -LRB- s -RRB- i , where i : f1 ; : : : ; sg ! f1 ; : : : ; jXjg is injective and monotonically increasing . We say that a subsequence hxi -LRB- 1 -RRB- ; : : : ; xi -LRB- s -RRB- i is an up-sequence if xi -LRB- 1 -RRB- < xi -LRB- 2 -RRB- < : : : < xi -LRB- s -RRB- . Similarly , we say that a subsequence hxi -LRB- 1 -RRB- ; : : : ; xi -LRB- s -RRB- i is a down-sequence if xi -LRB- 1 -RRB- > xi -LRB- 2 -RRB- > : : : > xi -LRB- s -RRB- . A subsequence is monotone if it is either a down-sequence or an up-sequence . We say that two subsequences hxi -LRB- 1 -RRB- ; : : : ; xi -LRB- s -RRB- i , hxj -LRB- 1 -RRB- ; : : : ; xj -LRB- t -RRB- i intersect if fi -LRB- 1 -RRB- ; i -LRB- 1 -RRB- +1 ; : : : ; i -LRB- s -RRB- g \ fj -LRB- 1 -RRB- ; j -LRB- 1 -RRB- +1 ; : : : ; j -LRB- t -RRB- g < > ; . A sequence X is a zig-zag shuffle if there is a partition of X into SMS -LRB- X -RRB- monotone subsequences such that no up-sequence intersects a down-sequence . For example , if W0 = h6 ; 5 ; 8 ; 7 ; 10 ; 9 ; 4 ; 3 ; 2 ; 1i , the subsequences h6 ; 8 ; 10i and h5 ; 7 ; 9i intersect but h6 ; 8 ; 10i and h4 ; 3 ; 2 ; 1i do not ; thus , W0 is not a zig-zag shuffle . We can obtain a geometric interpretation of zig-zag shuffles as follows : Given a sequence X , for each xi , we plot a point -LRB- i ; rank -LRB- xi -RRB- -RRB- in the plane , where rank -LRB- xi -RRB- is the position of xi when X is sorted . Given the 6 Estivill-Castro and Wood s s s s s s ss s s s s s s s s s s s s s s s s s s s ss s ss s s s s s s ???? ???????? ????????? -LRB- -LRB- -LRB- -LRB- -LRB- CCCXXXCCC @@ PPPAAAPPP LLLLJJJQQQ ??? ??? ??? aeaeae ????????? ???????? < = < = < = < = Figure 2 : A plot of a zig-zag shuffle . minimum decomposition of X into monotone subsequences , we join the consecutive points of each monotone subsequence with line segments . Then , X is a zig-zag shuffle if no down-sequence curve intersects an up-sequence curve ; for example , see Figure 2 . Zig-zag shuffles are important because if X is a zig-zag shuffle and SMS -LRB- X -RRB- < = z , then Enc -LRB- X -RRB- < = z and Melsort sorts X in O -LRB- jX j -LRB- 1 + log -LSB- z + 1 -RSB- -RRB- -RRB- time ; hence , Melsort can be used to sort zig-zag shuffles . Moreover , for any X , if Melsort attempts to construct an encroaching set with more than z sorted lists , then we have discovered in O -LRB- jX j -LRB- 1 + log -LSB- z + 1 -RSB- -RRB- -RRB- time that X is nonsimple . The partition protocol of Slab Sort first finds a value z such that SMS -LRB- X -RRB- < = z < = -LRB- SMS -LRB- X -RRB- -RRB- 2 , using O -LRB- jX j -LRB- 1 + log -LSB- z + 1 -RSB- -RRB- -RRB- comparisons , we give the details of this procedure later . Next , the partition protocol constructs a partition of the input sequence X by stably partitioning it into p = dz2 = 2e parts of almost equal size using the b1 + jX j = pc-th , b1 +2 jX j = pc-th , : : : , b1 + -LRB- p ? 1 -RRB- jX j = pcth elements as pivots . These elements are found by repeated median finding that make a total of O -LRB- jX j log -LSB- p +1 -RSB- -RRB- comparisons . The partitioning is similar to the partitioning in Quicksort , but with p ? 1 pivots , and it can be carried out in O -LRB- jX j -LRB- 1 + log -LSB- p + 1 -RSB- -RRB- -RRB- time , as is shown by Frazer and Mckellar for the partitioning in Samplesort -LSB- 6 -RSB- . To apply Theorem 3.1 , we must ensure that a constant fraction of the elements in X belong to a simple subsequence . Let X be a sequence with SMS -LRB- X -RRB- < = z. Assume that a minimum partition by monotone sequences has z1 down-sequences and z2 up-sequences with z1 + z2 < = z. Thus , the number of intersections among up-sequences and down-sequences is bounded by z1z2 < = z2 = 4 . Since Slab Sort stably partitions X into p = dz2 = 2e parts X1 ; : : : ; Xp , at least half of these parts are zig-zag shuffles with SMS -LRB- Xi -RRB- < = z. Hence , at least half of the parts are simple and , since all parts are almost the same size , at least half of the elements in the original sequence belong to simple sequences . We now explain how the partition protocol obtains a value z such that SMS -LRB- X -RRB- < = z < = -LRB- SMS -LRB- X -RRB- -RRB- 2 . The partition protocol starts with a guess of z = 2 , partitions X into p = dz2 = 2e parts , and tests whether half of these parts are simple using Melsort . If this condition is met , the partition protocol continues to construct the partition ; otherwise , it squares its guess of z and repeats the test . 5 Randomized Partition Sort In contrast to the pessimistic view taken by a worst-case analysis , expected-case analysis provides a more practical view , because worst-case situations are normally unlikely . There are two approaches to expected-case complexity -LSB- 7 -RSB- , the distributional approach and the randomized approach . The distributional approach may be inaccurate , since the probabilistic assumptions needed to carry out the analysis may be false and do not lead to useful notions of adaptivity -LSB- 5 -RSB- . The behavior of randomized sorting algorithms is , however , independent of the distribution of the instances to be sorted and they can be adaptive to the disorder in the input . Moreover , with randomization , we can often simplify sorting algorithms and their data structures ; for example , we give a version of Slab Sort that does not require median finding . We define Randomized Partition Sort , a randomized version of Partition Sort . It is based on An Adaptive Generic Sorting Algorithm that Uses Variable Partitioning 7 Randomized Partition Sort -LRB- X -RRB- X is sorted . Terminate . X is simple . Sort it using an algorithm for simple sequences . If X is neither sorted nor simple . Using a randomized partition protocol construct a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg : For i = 1 ; : : : ; kP -LRB- X -RRB- k , sort Xi recursively . Merge the sorted sequences to give X in sorted order . Figure 3 : A relaxed , randomized partition protocol results in Randomized Partition Sort . divide-and-conquer with a randomized partition protocol . We relax the requirement , however , that deeper recursive levels have no more parts than the partition at the first level . For a given sequence X , the randomized partition protocol obtains a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg with probability Pr -LSB- P -LRB- X -RRB- -RSB- . Each simple part Xi is sorted using an alternative sorting algorithm for simple sequences , and each part that is not simple is sorted by a recursive call . In the final step , all sorted parts are merged ; see Figure 3 . The number of parts kP -LRB- X -RRB- k in the partition is a random variable ; thus , the expected time taken for the merge is no more than X P -LRB- X -RRB- ? Pr -LSB- P -LRB- X -RRB- -RSB- jX j -LRB- 1 + log -LSB- kP -LRB- X -RRB- k + 1 -RSB- -RRB- < = jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- ; where > is a constant and E -LSB- kP -LRB- X -RRB- k -RSB- is the expected value of kP -LRB- X -RRB- k. Now , if the expected time taken by the partition protocol to obtain a partition of X and the expected time taken by the recursive calls are bounded from above by cjX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- +1 -RSB- -RRB- , then we obtain an algorithm that is adaptive with respect to E -LSB- kP -LRB- X -RRB- k -RSB- . This approach works , however , only if the expected length of the simple sequences is a constant fraction of the total length of the sequence to be partitioned . Theorem 5.1 Let c ; d and k be constants with < c < = 1 , d > o and k > . Let RDP be a randomized partition protocol such that , for all sequences X 2 N < N : 1 . If X is nonsimple , then the RDP realizes this and creates a partition P -LRB- X -RRB- = fX1 ; : : : ; XkP -LRB- X -RRB- kg of X in no more than djX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- comparisons ; 2 . For all parts of all partitions , E -LSB- kP -LRB- Xi -RRB- k -RSB- < = E -LSB- kP -LRB- X -RRB- k -RSB- ; 3 . If J -LRB- P -LRB- X -RRB- -RRB- is the set of indices of the simple parts , then cjX j < = X P -LRB- X -RRB- Pr -LSB- P -LRB- X -RRB- -RSB- X i2J -LRB- P -LRB- X -RRB- -RRB- jXij ; 4 . There is a sorting algorithm S that sorts any simple sequence Xi in P -LRB- X -RRB- , making no more than kjXij -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- comparisons . Then , the expected number of comparisons to sort X by Randomized Partition Sort is O -LRB- jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- +1 -RSB- -RRB- -RRB- : Randomized Partition Sort is adaptive with respect to the expected size of the initial partition . Proof : Let > be a constant such that the merging of the results of recursive calls takes no more than jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- time . Let u > -LRB- d + k + -RRB- = c and let E -LSB- TRPS -LRB- X -RRB- -RSB- denote the expected number of comparisons performed by Randomized Partition Sort on input 8 Estivill-Castro and Wood X . We prove by induction on jX j that E -LSB- TRPS -LRB- X -RRB- -RSB- < = jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- : We present only the induction step since the basis is immediate . We assume that if Z is a sequence with jZj < jX j , then E -LSB- TRPS -LRB- Z -RRB- -RSB- < = jZj -LRB- 1 + log -LSB- E -LSB- kP -LRB- Z -RRB- k -RSB- + 1 -RSB- -RRB- : Now , E -LSB- TRPS -LRB- X -RRB- -RSB- is the sum of four quantities . The first is the expected number of the comparisons used to partition X and it is bounded by djX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- : The second is the expected number of comparisons used to identify and sort the simple parts and it is bounded by kjX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- : The third is the expected number of comparisons used to merge the sorted parts and it is bounded by jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- : The fourth is the expected number of comparisons used by the recursive calls on the nonsimple parts of the partition . Let Xi , for i = 1 ; : : : ; kP -LRB- X -RRB- k ; be the parts of X with respect to the partition P -LRB- X -RRB- , and let J -LRB- P -LRB- X -RRB- -RRB- = fi j 1 < = i < = kP -LRB- X -RRB- k and Xi is simpleg : Thus , by the induction hypothesis , E -LSB- TRPS -RSB- < = -LRB- d + k + -RRB- jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- + X P -LRB- X -RRB- X i62J -LRB- P -LRB- X -RRB- -RRB- E -LSB- TRPS -LRB- Xi -RRB- -RSB- < = -LRB- d + k + -RRB- jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- + u -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- X i62J -LRB- P -LRB- X -RRB- -RRB- E -LSB- kXik -RSB- < = -LRB- d + k + + -LRB- 1 ? c -RRB- u -RRB- jX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- < = ujX j -LRB- 1 + log -LSB- E -LSB- kP -LRB- X -RRB- k -RSB- + 1 -RSB- -RRB- ; which completes the induction step . 2 6 An Application of Randomized Partition Sort We apply Randomized Partition Sort to obtain Randomized Slabsort , a randomized version of Slab Sort . Rather than finding elements that partition the input sequence X into parts of almost equal length , a sample of p = dk2 = 2e elements is chosen at random . These elements are used as pivots to stably partition X . This partition protocol is analogous to Samplesort -LSB- 6 -RSB- . Samplesort is a generalization of Randomized Quicksort in which , rather than choosing a single pivot at random , a sample of several pivots is chosen at random . To select any subset of p elements with equal probability , the sample is selected as described by Knuth -LSB- 8 , Algorithm S , page 137 -RSB- . Thus , the stable partitioning can be carried out with O -LRB- jX j -LRB- 1 + log -LSB- p + 1 -RSB- -RRB- -RRB- comparisons and with a much smaller multiplicative constant than with median finding ; however , the length of the i-th part is now a random variable with expectation -LRB- jX j ? p -RRB- = -LRB- p + 1 -RRB- -LSB- 6 -RSB- . Using Theorem 5.1 , we now show that Randomized Slabsort achieves the same asymptotic complexity as Slab Sort , but in the expected case . Since we do not know the lengths of the parts that are successfully sorted by Melsort , the analysis is different from the analysis of Samplesort -LSB- 6 -RSB- and Slab Sort -LSB- 10 -RSB- . We verify the conditions of Theorem 5.1 . Let LZ -LRB- X -RRB- be the sum of the lengths of the parts that are zig-zag shuffles and let LN -LRB- X -RRB- be the sum of the lengths of the parts that are not zig-zag shuffles . LZ -LRB- X -RRB- and LN -LRB- X -RRB- are random variables over all partitions of X into p parts with LZ -LRB- X -RRB- + LN -LRB- X -RRB- = jX j. Moreover , LZ -LRB- X -RRB- is always larger than the sum of the lengths of the bp = 2c smallest parts . Let L -LRB- X -RRB- denote the length of the bp = 2c smallest parts . The expected length of the i-th smallest part , when partitioning X into p pieces , is jX j p iX j = 1 1 p ? -LRB- j ? 1 -RRB- -LSB- 12 , page 66 -RSB- ; thus , E -LSB- L -LRB- X -RRB- -RSB- = bp = 2c X i = 1 jX j p iX j = 1 1 p ? -LRB- j ? 1 -RRB- : An Adaptive Generic Sorting Algorithm that Uses Variable Partitioning 9 Now , let Hn = Pni = 1 1 = i denote the n-th harmonic number . Algebraic manipulation shows that E -LSB- L -LRB- X -RRB- -RSB- = jX j -LRB- Hbp = 2c ? Hp + 1 -RRB- = 2 ; therefore , E -LSB- L -LRB- X -RRB- -RSB- > = -LRB- 1 ? ln -LRB- 2 -RRB- -RRB- jX j = 2 and we have proved the following result . Lemma . Let X 2 N < N and S be a uniform sample of p elements . Then , there is a constant c0 such that < c0 < 1 and the expected length of the smallest bp = 2c parts is at least c0jX j. Therefore , there is a constant < c0 < 1 such that E -LSB- LZ -LRB- X -RRB- -RSB- > = c0jX j and E -LSB- LN -LRB- X -RRB- -RSB- < = -LRB- 1 ? c0 -RRB- jX j ; which allows us to apply Theorem 5.1 to obtain the following result . Theorem 6.1 Let TS -LRB- X -RRB- denote the number of comparisons taken by Randomized Slabsort -LRB- X -RRB- . Then , there is a constant d > such that , for all sequences X 2 N < N , E -LSB- TS -LRB- X -RRB- -RSB- < = djX j -LRB- 1 + log -LSB- SMS -LRB- X -RRB- + 1 -RSB- -RRB- : Thus , we have obtained a sorting algorithm that is optimally adaptive for SMS , SUS , Runs , Dis and Max and does not use median finding . 7 Final Remarks There are many measures of disorder ; for example , the number of operations that are required to rearrange a sequence into sorted order may be our first concern . We define Exc as the minimum number of exchanges that are required to sort a sequence . We may also consider that disorder is produced by the insertion of some records into the wrong positions . We define Rem as the minimum number of elements that must be removed to obtain a sorted subsequence . A sorting algorithm that is optimally adaptive for SMS is also optimally adaptive for Runs , Max , Dis , and SUS -LSB- 10 -RSB- , but , by combining partition protocols , we can also achieve optimality for Inv , Exc and Rem -LSB- 5 -RSB- . This result illustrates the power of our approach : concentrating on design of partition protocols and , then , alternating them to obtain adaptivity for more measures . All known adaptive sorting algorithms that are based on divide-and-conquer achieve adaptivity by choosing an appropriate partition protocol in the division step . An interesting open question is : can we achieve adaptivity by choosing an appropriate method of combining subsequences ? For example , we might assume that subproblems are only nearly sorted . In some sense , such approach is used in bitonic sort -LSB- 9 -RSB- , where bitonic sequences are merged to give sorted sequences . References -LSB- 1 -RSB- T. Altman and Y. Igarashi . Roughly sorting : Sequential and parallel approach . Journal of Information Processing , 12:154 -LCB- 158 , 1989 . -LSB- 2 -RSB- J. Chen and S. Carlsson . On partitions and presortedness of sequences . In Proceedings of the 2nd ACM-SIAM Symposium on Discrete Algorithms , pages 63 -LCB- 71 , San Francisco , CA , 1991 . -LSB- 3 -RSB- V. Estivill-Castro and D. Wood . A new measure of presortedness . Information and Computation , 83:111 -LCB- 119 , 1989 . -LSB- 4 -RSB- V. Estivill-Castro and D. Wood . A generic adaptive sorting algorithm . Computer Journal , 35:505 -LCB- 512 , 1992 . -LSB- 5 -RSB- V. Estivill-Castro and D. Wood . A survey of adaptive sorting algorithms . Computing Surveys , 24:441 -LCB- 476 , 1992 . -LSB- 6 -RSB- W.D. Frazer and A. C. McKellar . Samplesort : A sampling approach to minimal storage tree sorting . Journal of the ACM , 17 -LRB- 3 -RRB- :496 -LCB- 507 , July 1970 . -LSB- 7 -RSB- R. M. Karp . Combinatorics , complexity and randomness . Communications of the ACM , 29 -LRB- 2 -RRB- :98 -LCB- 109 , February 1986 . 10 Estivill-Castro and Wood -LSB- 8 -RSB- D.E. Knuth . The Art of Computer Programming , Vol .2 : Seminumerical Algorithms . AddisonWesley Publishing Co. , Reading , MA , 1973 . -LSB- 9 -RSB- D.E. Knuth . The Art of Computer Programming , Vol .3 : Sorting and Searching . AddisonWesley Publishing Co. , Reading , MA , 1973 . -LSB- 10 -RSB- C. Levcopoulos and O. Petersson . Sorting shuffled monotone sequences . In J.R. Gilbert and R. Karlsson , editors , Proceedings of the 2nd Scandinavian Workshop on Algorithm Theory , pages 181 -LCB- 191 , Bergen , Sweden , 1990 . Springer-Verlag Lecture Notes in Computer Science 447 . -LSB- 11 -RSB- H. Mannila . Measures of presortedness and optimal sorting algorithms . IEEE Transactions on Computers , C-34 :318 -LCB- 325 , 1985 . -LSB- 12 -RSB- F. Mosteller . Fifty Challenging Problems in Probability with Solutions . Addison-Wesley Publishing Co. , Reading , MA , 1965 . -LSB- 13 -RSB- S.S. Skiena . Encroaching lists as a measure of presortedness . BIT , 28:755 -LCB- 784 , 1988 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O