On the Scalability of Parallel Quicksort : A Case Study on Distributed vs . Shared-Memory Models

POSTER PAPER — The increasing availability of parallel systems affords undergraduates opportunities to experience firsthand the potential benefit and pitfalls of parallel programming. However, a clear understanding of the underlying architecture is critical to achieve the anticipated speed up of serial programs. To this end, analyzing performance metrics is essential for tuning parallel implementations and determining what improvement can be expected by scaling out or increasing the number of processes [1-4]. Without considering important aspects such locality, load balancing and communication overheads associated with each available parallel system, students are having a hard time obtaining the intended performance and determining which approach is suitable to solving the problem at hand. Finding the best mapping between the parallel implementation and the parallel platform available is hard to achieve without clear guidance or practical experiences. Given some legacy code, a first approach consists of looking at parallel programming patterns such as divide-and-conquer or decomposition techniques. This pattern has been used widely to design a number of efficient algorithms for a variety of applications [5]. Students are already familiar with the approach of recursively partitioning a problem into smaller sub-problems. Identification of sub-problems which can be solved independently is an intuitive choice for achieving speed up. Even if this is not always the case, this approach allows students to quickly implement embarrassingly parallel programs using message passing and shared-address space programming paradigms. Ultimately, performance metrics are collected and contrasted with the serial and parallel runtime implementations. Sources of overhead affecting the performances of the algorithms are examined and discussed. This practical approach can be repeated on various classical problems (matrix multiplication, sorting, numerical integration and the gravitational N-body) and extended to new problems by applying various parallel programming patterns This case study walks through the steps of solving the problem of speeding up sorting serial problems from legacy code for both shared memory and distributed memory models on a small system multi-core cluster called Bucc. Bucc is a small cluster available to undergraduate students at the University of the Virgin Islands who are exposed to highperformance computing applications and are interested in solving computational science problems. Bucc consists of a head node with 2 Intel Westmere, 2.40GHz, six-core, processors and five twin compute nodes, each with two Intel 2.40GHz six-core processors. It is configured with 2 GB of RAM per core. The interconnects are InfiniBand 40Gbps. This study focuses specifically on the sorting task for both the distributed and shared-memory models on the Bucc system. Sorting is an recurrent critical problem for organizing and processing efficiently information in many fields such as combinatorial searching, optimization, simulation, information retrieval, scientific computing etc. Sorting is also used as a core utility in many libraries. Because of its essential role computer scientists have devoted many studies on sorting techniques and have been focusing on finding the fastest serial and parallel implementations. The quicksort algorithm is one of the most popular algorithms known for its time complexity of O(N log N). As a divide-and-conquer algorithm, it uses the notion of pivoting to divide the data set into two halves or subsets that are recursively solved; there are several ways of determining the pivot such as finding the median, selecting the middle value, or simply selecting a random value in the data set. Selecting the pivot is important as it ultimately affects the time taken to sort and merge all the results together. The quicksort algorithm has the worst case of being O(N), which can render it as slow as the selection and bubble sorting algorithms. Due to its divide-and-conquer nature, intuitive parallel implementations are usually implemented using message passing interface libraries (MPI) and shared-addess space libraries (OpenMP). Other hybrid programming models are also considered but not examined in this study. The naive parallel formulation of the serial quicksort consists of sorting a global array A of n element on p processes. Each process is given a sub-array which is a sequence of n/p items. The sub-array is partitioned recursively around a pivot using a compare and split routine. The algorithm terminates when the sub-array can not be partitioned anymore. Fig. 1 show the serial quicksort runtime of Bucc. The performance and scalability of parallel programs (MPI and OpenMP) are discussed. OpenMP quicksort spawns t threads each of them handle a partition of the array. Each thread uses the serial qsort() from the C libraries [6]. Once all partitions have been sorted, the sub-arrays are then merged together. The Int'l Conf. Par. and Dist. Proc. Tech. and Appl. | PDPTA'15 | 335