Abstract ? Given a parallel program represented by a task graph , the objective of a scheduling algorithm is to minimize the overall execution time of the program by properly assigning the nodes of the graph to the processors . In this paper we propose a novel GA-based algorithm with an objective to simultaneously meet the goals of high performance , scalability , and fast running time . The proposed PGS -LRB- Parallel Genetic Scheduling -RRB- algorithm itself is a parallel algorithm and generates high quality solutions in a short time . By encoding a scheduling list as a chromosome , the PGS algorithm can potentially generate an optimal scheduling list which in turn leads to an optimal schedule . The major strength of the PGS algorithm lies in its two efficient genetic operators : the order crossover and mutation which effectively combine the building-blocks of good scheduling lists to construct better lists . When evaluated with random task graphs for which optimal solutions are known , the algorithm generates optimal solutions for more than half of the test cases and close-to-optimal for the remaining cases . 1 Introduction Given a parallel program modeled by a node - and edgeweighted directed acyclic graph -LRB- DAG -RRB- , finding an optimal schedule with a minimum turnaround time is well known to be an NP-complete problem -LSB- 2 -RSB- , -LSB- 3 -RSB- . The problem can be solved in a polynomial-time only for a few simplified cases -LSB- 2 -RSB- , -LSB- 3 -RSB- . If the simplifying assumptions of these cases are relaxed , the problem becomes NP-hard in the strong sense . Therefore , state-space search techniques are considered to be the only resort for finding optimal solutions -LSB- 7 -RSB- . However , most of these techniques are still designed to work under restricted environments and are not applicable to general situations . Due to the intractability of the scheduling problem , many polynomial-time heuristics are reported to tackle the problem under more pragmatic situations -LSB- 1 -RSB- , -LSB- 5 -RSB- , -LSB- 6 -RSB- , -LSB- 9 -RSB- , -LSB- 13 -RSB- , -LSB- 14 -RSB- . While these heuristics are shown to be effective in experimental studies , they usually can not generate optimal solutions , and there is no guarantee in their performance in general . Furthermore , these heuristics are usually not scalable in terms of their running times and solution quality for large problem sizes . In view of the drawbacks of the existing sequential ? . This research was supported by the Hong Kong Research Grants Council under contract number HKUST 734/96E . scheduling heuristics , we aim at designing a new scheduling scheme that has a high capability to generate optimal solutions and is also fast and scalable . To obtain high quality solutions , we devise a genetic formulation of the scheduling problem , in which scheduling lists -LRB- ordering of tasks for scheduling -RRB- are systematically combined by using computationally efficient operators so as to determine an optimal scheduling list . To achieve a reduced time complexity , the proposed algorithm is parallelized . The algorithm scales well with the number of processors . Moreover , the algorithm considers the problem for general DAGs , without making simplifying assumptions . The remainder of the paper is organized as follow . In the next section we provide the problem statement . In Section 3 we present the proposed parallel genetic scheduling algorithm . Examples are used to illustrate the functionality of the proposed technique . In Section 4 we describe our experimental study and its results . Finally , we provide some concluding remarks in the last section . 2 Problem Statement In static scheduling , a parallel program can be modeled by a directed acyclic graph -LRB- DAG -RRB- , where V is a set of v nodes and E is a set of e directed edges . A node in the DAG represents a task which in turn is a set of instructions that must be executed sequentially without preemption in the same processor . The weight associated with a node , which represents the amount of time needed for a processor to execute the task , is called the computation cost of a node and is denoted by . The edges in the DAG correspond to the communication messages and precedence constraints among the nodes . The weight associated with an edge , which represents the amount of time needed to communicate the data , is called the communication cost of the edge . The communication-to-computation-ratio -LRB- CCR -RRB- of a parallel program is defined as its average communication cost divided by its average computation cost on a given system . An example DAG is shown in Figure 1 . The target multiprocessor system is modeled as a fully-connected network of processing elements -LRB- PEs -RRB- . The objective of scheduling is to minimizing the schedule length , which is defined as the maximum finish time among all tasks . 3 The Proposed Parallel Genetic Algorithm for Scheduling Genetic algorithms are intelligent search techniques which have found many applications in optimization problems -LSB- 4 -RSB- , -LSB- 11 -RSB- . In simple terms , given a well-defined search-space in which each point is represented by a bit string called a chromosome , a GA is applied by using three genetic G V E , -LRB- -RRB- = ni w n i -LRB- -RRB- A Parallel Genetic-Search-Based Algorithm for Scheduling Arbitrary Task Graphs to Multiprocessors Yu-Kwong Kwok and Ishfaq Ahmad Department of Computer Science The Hong Kong University of Science and Technology , Hong Kong Email : -LCB- csricky , iahmad -RCB- @cs . ust.hk search operators selection , crossover , and mutation , to improve the quality of a population of potential solutions . Due to its recombinative nature in the searching process , a GA can be very effective in finding optimal solutions -LSB- 4 -RSB- . The proposed genetic formulation of the scheduling problem is as follows . Encoding : We make a valid scheduling list as a chromosome . A valid scheduling list is one in which the nodes of the DAG are in a topological order . Fitness of a Chromosome : Fitness value is defined as : , where the schedule length SL is determined by using the start-time minimization method . Thus , the fitness of a chromosome is always bounded between 0 and 1 . Generation of the Initial Population : An initial population is generated from a set of scheduling lists which are constructed by ALAP ordering , b-level ordering , t-level ordering , static-level ordering and a random topological ordering . These different orderings not only provide the necessary diversity yet also represent a population with a higher fitness than a set of totally random topological orderings . The whole population is then generated from these orderings by performing random swapping of nodes . 3.1 Genetic Operators As the standard crossover and mutation operators may violate precedence constraints , we need to use some other well-defined genetic operators . The inversion operator is not considered because it can obviously generate invalid scheduling lists . We use the order crossover -LSB- 4 -RSB- . We also describe a a mutation operator based on node-swapping . Order Crossover Operator : We consider a single-point order crossover operator . That is , given two parents , we first pass the left segment -LRB- i.e. , the segment on the left of the crossover point -RRB- from the first parent , called parent 1 , to the child . Then we construct the right fragment of the child by taking the remaining nodes from the other parent , called parent 2 , in the same order . An example of the crossover operator is given in Figure 2 -LRB- a -RRB- . Note that the chromosomes shown in the figure are all valid topological ordering of the DAG in Figure 1 -LRB- a -RRB- . The left segment of parent 1 is passed directly to the child . Then the nodes in the right segment of parent 1 are appended to the child according to their order in parent 2 . This order Figure 1 : An example DAG . n12 n2 3 n5 5 n9 n3 3 n7 4 n4 4 n8 4 n6 4 1 6 10 1 1 11 4 1 1 5 5 1 w n i -LRB- -RRB- SL ? ? -LRB- -RRB- w ni -LRB- -RRB- ? ? n 1 n2 n7 n 4 , , , -LCB- -RCB- n3 n 8 n6 n9 n 5 , , , , -LCB- -RCB- crossover operator is easy to implement and permits fast processing . The most important merit is that it never violates the precedence constraints , as dictated by following theorem . Theorem 1 : The order crossover operator always produces a valid scheduling list from two valid parent chromosomes . Proof : Suppose we are given two parent chromosomes and . Let the crossover point be chosen at the kth position . Then after applying the order crossover , the child will be for some indices x and y. Obviously the precedence constraints among any two nodes at or before the kth position will be respected . For a node at or before the kth position and a node after the kth position , the precedence constraint -LRB- if any -RRB- will also be respected because their relative positions are the same as in parent 1 . Finally for any two nodes after the kth position , the precedence constraint -LRB- if any -RRB- will also be respected because their relative positions are the same as in parent 2 , by definition of the order crossover operator . -LRB- Q.E.D. -RRB- Mutation Operator : We define the mutation operator as a swapping of two nodes in the list . A valid topological order can be transformed into another topological order by swapping some nodes . Not every pairs of nodes can be swapped without violating the precedence constraints . Two nodes are interchangeable if they are not lying on the same path in the DAG . Using a pre-processing depth-first traversal of the DAG , we can check whether two nodes are interchangeable in a constant time during the search . An example is given in Figure 2 -LRB- b -RRB- where two interchangeable nodes and are randomly chosen and their positions in the list are swapped . 3.2 Control Parameters We use adaptive control parameters as suggested by Srinivas et al. . -LSB- 10 -RSB- . The adaptive crossover rate is defined as follows : where is the maximum fitness value in the local population , is the average fitness value , is the fitness value of the fitter parent for the crossover , and is a positive real constant less than 1 . The adaptive mutation rate is parent 1 parent 2 crossover point Figure 2 -LRB- a -RRB- : An example of the order crossover operator . Mutated -LRB- swapped and -RRB- n 4 n 7 Figure 2 -LRB- b -RRB- : An example of the mutation operator . -LCB- n1 , n2 , n7 , n4 , n3 , n8 , n6 , n9 , n5 -RCB- -LCB- n1 , n4 , n2 , n3 , n7 , n6 , n8 , n5 , n9 -RCB- -LCB- n1 , n2 , n7 , n4 , n3 , n6 , n8 , n5 , n9 -RCB- -LCB- n1 , n2 , n7 , n4 , n3 , n8 , n6 , n9 , n5 -RCB- -LCB- n1 , n2 , n4 , n7 , n3 , n8 , n6 , n9 , n5 -RCB- n i1 ni2 ? niv , , , -LCB- -RCB- nj 1 nj2 ? n jv , , , -LCB- -RCB- ni1 n i2 ? n i k n jx ? nj y , , , , , , -LCB- -RCB- n 4 n7 m c mc kc fm a x f ?? -LRB- -RRB- fm a x fav g ? -LRB- -RRB- ------------------------------- = f max f a vg f ? kc mc defined as follows : where f is the fitness value of the chromosome to be mutated and is a positive real constant less than 1 . Using the above adaptive crossover and mutation rate , the best chromosome is protected from disruption by crossover and mutation . On the other hand , when the population tends to be become more homogeneous , both rates increase because will be about the same as . Thus , under such a situation , chromosomes are more likely to be perturbed . Note that even though the initial setting of the crossover rate and mutation rate is the same for all the parallel processors , the adaptive strategy gradually leads to the desired heterogeneity of the parameters among the processors . Two other control parameters which are critical to the performance of a GA are the population size and the number of generation . Usually and are fixed for all problem sizes . However , this is not appropriate because larger problems require more time for exploration and exploitation . We , therefore , vary these two parameters linearly according to the problem size . Specifically , we set and , where and are real constants . 3.3 Parallelization The global population is partitioned into q subpopulations , where q is the number of PPEs . For efficiency we use a synchronous connected island model -LSB- 8 -RSB- , -LSB- 12 -RSB- , in which PPEs communicate periodically to exchange the fittest individual and the communication is a synchronous voting such that the fittest individual is broadcast to all the PPEs . When the PPEs communicate , only the best chromosome migrates . When the fittest chromosome is imported , the worst chromosome in the local population is discarded . Then the fittest chromosome and the locally best chromosome are protected from the rank-based selection process . The period of communication for the PPEs is set to be T number of generations , which follows an exponentially decreasing sequence : initially , then , , and so on . The rationale is that at the beginning of the search , the diversity of the global population is high . At such early stages , exploration is more important than exploitation and , therefore , the PPEs should work on the local sub-population independently for a longer period of time . When the search reaches the later stages , it is likely that the global population converges to a number of different fittest chromosomes . Thus , exploitation of more promising chromosomes is needed to avoid unnecessary work on optimizing the locally best chromosomes that may have smaller fitness values than the globally best chromosomes . With the above design considerations , the Parallel Genetic Scheduling -LRB- PGS -RRB- algorithm is outlined below . mm km fm a x f ? -LRB- -RRB- fm a x favg ? -LRB- -RRB- ------------------------------- = km fav g fm a x Np Ng Np Ng Np kpv = Ng k g v = k p k g v 2 -- v 4 -- v 8 -- Parallel Genetic Scheduling -LRB- PGS -RRB- : -LRB- 1 -RRB- Generate a local population with size equal to by perturbing a pre-defined topological ordering of the DAG -LRB- e.g. , ALAP ordering , b-level ordering , etc. -RRB- . -LRB- 2 -RRB- ; -LRB- 3 -RRB- repeat -LRB- 4 -RRB- ; -LRB- 5 -RRB- repeat -LRB- 6 -RRB- Using the current crossover and mutation rates , applied the operators to the local population . -LRB- 7 -RRB- until + + T = ; -LRB- 8 -RRB- Accept the best chromosome from a remote PPE and discard the worst local chromosome accordingly . -LRB- 9 -RRB- Explicitly protect the best local and remote chromosome and adapt new crossover and mutation rates . -LRB- 10 -RRB- -LRB- 11 -RRB- until total number of generations elapsed equal to ; 4 Performance Results To examine the efficacy of the PGS algorithm , we have implemented it on the Intel Paragon using the C language and tested it using a set of random task graphs , for which the optimal solutions are known . The suite of graphs consists of three sets of randomly generated task graphs with different CCRs : 0.1 , 1.0 , and 10.0 . Each set consists of 12 graphs in which the number of nodes vary from 10 to 32 with increments of 2 . The graphs within the same set have the same value of CCR . To obtain optimal solutions for the task graphs , we applied a parallel A * algorithm to the graphs -LSB- 7 -RSB- . Since generating optimal solutions for arbitrarily structured task graphs takes exponential time , it is not feasible to obtain optimal solutions for larger graphs . Since the performance of a genetic algorithm critically depends on the population size and number of generations used , we performed experiments to determine the most suitable values for these control parameters . However , due to space limitations , we do not present those results in this paper . Based such results , in the subsequent experiments and were fixed as 5 and 10 , respectively . That is , and . In the experiment we aimed to investigate the effect of the number of PPEs on the performance of the algorithm . We applied the PGS algorithm to the random task graphs on the Paragon using 2 , 4 , 8 , and 16 PPEs . The percentage deviations from optimal solutions for the three values of CCR were noted and are shown in Table 1 . In the table , the total number of optimal solutions generated and the average percentage deviations for each CCR are also shown . These results indicate that the PGS algorithm demonstrated similar performance for 2 , 4 , and 8 PPEs . When 16 PPEs were used , the performance degraded by a moderately larger margin . One explanation for this phenomenon is that using more PPEs implies the size of a local population is smaller . This in turn leads to smaller diversity of the local population . Premature convergence is then more likely to result . The effect of CCR was quite considerable in that the average deviations increase with increasing CCRs . Np q ? i 2 = T = Ng i ? i i 2 ? = Ng kp k g Np 5v = N g 10v = To examine the effectiveness of parallelization , we observed the execution times of the PGS algorithm using different number of PPEs . As a reference , we also ran the PGS algorithm using 1 PPE on the Paragon . The average execution times and speedups are shown in Figure 3 . As can be seen from the speedup plot , the PGS algorithm demonstrated a slightly less than linear speedup . This is because the migration of best chromosomes contributed considerable communication overhead . From the above results we find that the number of generations used critically affects the performance of the PGS algorithm . In view of this , we ran the algorithm with larger 101214161820222426283032 Graph Size Table 1 : Results of the PGS algorithm compared against optimal solutions -LRB- % deviations -RRB- for the random task graphs with three CCRs using 2 , 4 , 8 , and 16 PPEs on the Intel Paragon . CCR No . of PPEs 0.00 0.00 0.00 7.96 0.00 0.00 0.00 5.05 0.00 0.00 0.00 9.28 7.59 6.71 6.17 16.28 7.98 7.23 7.23 7.72 0.00 0.00 0.00 0.26 13.02 12.09 11.99 24.40 6.16 5.35 5.21 12.02 10.23 10.19 9.65 32.50 0.00 0.00 0.00 10.29 2.30 2.25 2.09 2.33 1.20 1.05 1.03 1.98 2 4 8 16 10.0 0.00 0.00 0.00 3.14 0.00 0.00 0.00 16.35 0.00 0.00 0.00 5.41 9.23 8.05 7.87 17.78 0.00 0.00 0.00 5.23 0.00 0.00 0.00 10.96 0.00 0.00 0.00 17.98 0.00 0.00 0.00 10.27 0.00 0.00 0.00 9.41 6.07 5.24 5.12 10.41 6.32 5.26 4.88 11.01 0.00 0.00 0.00 13.16 2 4 8 16 1.0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.32 0.00 0.00 0.00 0.00 0.86 0.62 0.60 0.67 6.15 5.18 5.01 5.99 0.00 0.00 0.00 5.30 0.00 0.00 0.00 4.57 4.01 3.31 2.89 5.08 6.57 5.30 5.04 14.58 4.66 3.20 3.11 7.16 4.27 2.59 2.16 4.53 3.34 2.31 2.11 3.30 2 4 8 16 0.1 No . of Opt . Avg . Dev . 5 5 5 6.93 6.41 6.20 10.84 9 9 9 7.21 6.18 5.96 10.93 5 5 5 2 4.27 3.22 2.99 5.35 10 12 14 16 18 20 22 24 26 28 30 32 2 4 6 8 10 12 14 16 2 PPEs 4 PPEs 8 PPEs 16 PPEs Figure 3 : -LRB- a -RRB- The average running times of the PGS algorithm for the random task graphs with three CCRs using 1 PPE on the Intel Paragon ; -LRB- b -RRB- The average speedups of the PGS algorithm for 2 , 4 , 8 , and 16 PPEs . 101214161820222426283032 Graph Size 4.12 4.64 4.86 5.02 5.32 5.79 6.03 6.34 6.89 7.12 7.63 7.82 Running Times -LRB- secs -RRB- 2 5 10 20 30 40 50 5 10 15 20 25 CCR = 0.1 CCR = 1.0 CCR = 10.0 Figure 4 : The average % deviations from the optimal of the solutions generated by the PGS algorithm for the random task graphs with three CCRs using 8 PPEs on the Intel Paragon . number of generations to see how far it can approach optimal . We used 8 PPEs on the Paragon and other parameters remained the same . The results are shown in Figure 4 . Note that the results for the smaller number of generations are also included . We notice that when the number of generations is increased to , the PGS algorithm is almost optimal . 5 Conclusions We have presented a parallel genetic algorithm called the PGS algorithm for multiprocessor DAG scheduling . The major motivation of using a genetic search approach is that the recombinative nature of a genetic algorithm can potentially determine an optimal scheduling list leading to an optimal schedule . The PGS algorithm judiciously combines good building-blocks of scheduling lists to construct better lists . Parallelization of the algorithm is based on a novel approach in that the parallel processors communicate to exchange the best chromosomes with exponentially decreasing periods . Using random task graphs in the performance evaluation , the PGS algorithm generates optimal solutions for a major portion of the test cases . The PGS algorithm also demonstrates an almost linear speedup and is therefore scalable in that by using more parallel processors , the algorithm can be used for scheduling large task graphs . References -LSB- 1 -RSB- I. Ahmad and M.K. Dhodhi , ? Multiprocessor Scheduling in a Genetic Paradigm , ? Parallel Computing , vol . 22 , no. 3 , Mar. 1996 , pp. 395-406 . -LSB- 2 -RSB- E.G. Coffman , Computer and Job-Shop Scheduling Theory , Wiley , New York , 1976 . -LSB- 3 -RSB- M.R. Garey and D.S. Johnson , Computers and Intractability : A Guide to the Theory of NP-Completeness , 1979 . -LSB- 4 -RSB- D.E. Goldberg , Genetic Algorithms in Search , Optimization and Machine Learning , Addison-Wesley , Mass. , 1989 . -LSB- 5 -RSB- E.S.H. Hou , N. Ansari , and H. Ren , ? A Genetic Algorithm for Multiprocessor Scheduling , ? IEEE Trans . Parallel and Distributed Sys. , vol . 5 , no. 2 , Feb. 1994 , pp. 113-120 . -LSB- 6 -RSB- Y.-K . Kwok and I. Ahmad , ? Dynamic Critical-Path Scheduling : An Effective Technique for Allocating Task Graphs onto Multiprocessors , ? IEEE Transactions on Parallel and Distributed Systems , vol . 7 , no. 5 , May 1996 , pp. 506-521 . -LSB- 7 -RSB- Y.-K . Kwok , I. Ahmad , M. Kafil , and Imitaz Ahmad , ? Parallel Algorithms for Optimal Scheduling of Arbitrary Task Graphs to Multiprocessors , ? submitted for publication . -LSB- 8 -RSB- M. Rebaudengo and M. Sonza Reorda , ? An Experimental Analysis of the Effects of Migration in Parallel Genetic Algorithms , ? Proceedings of Euromicro Workshop on Parallel and Distributed Processing , 1993 , pp. 232-238 . -LSB- 9 -RSB- G.C. Sih and E.A. Lee , ? A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures , ? IEEE Trans . on Parallel and Distributed Systems , vol . 4 , no. 2 , pp.75-187 , Feb. 1993 . -LSB- 10 -RSB- M. Srinivas and L.M. Patnaik , ? Adaptive Probabilities of Crossover and Mutation in Genetic Algorithms , ? IEEE Trans . Sys. , Man and Cyb. , vol . 24 , no. 4 , Apr. 1994 , pp. 656-667 . -LSB- 11 -RSB- ? , ? Genetic Algorithms : A Survey , ? Computer , Jun. 1994 , pp. 17-26 . -LSB- 12 -RSB- R. Tanese , ? Parallel Genetic Algorithm for a Hypercube , ? Proc . Int?l Conf . on Genetic Alg , 1987 , pp. 177-183 . -LSB- 13 -RSB- L. Wang , H.J. Siegel , and V.P. Roychowdhury , ? A GeneticAlgorithm-Based Approach for Task Matching and Scheduling in Heterogeneous Computing Environments , ? Proc . 5th Heterogeneous Computing Workshop , 1996 , pp. 72-85 . -LSB- 14 -RSB- T. Yang and A. Gerasoulis , ? DSC : Scheduling Parallel Tasks on an Unbounded Number of Processors , ? IEEE Trans . Parallel and Dist . Sys. , vol . 5 , no. 9 , Sep. 1994 , pp. 951-967 . 40v

O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

genetic algorithms
multiprocessors
parallel algorithms
scheduling
task graph