Course 2 — Data structure — Part 2: Priority queues and Disjoint set | by Phat Le | Towards Data Science

If we ever want to know how background job works, fastest way to find k smallest elements in an array, how merging tables in database works behind the scenes, keep reading. Because in this article, we will discuss about priority queues and disjoint set. Both data structures are beautiful to solve these problems. In the end, we will solve these problems above. Happy reading!
Priority Queue is a Queue where each element is assigned a priority and elements com out in order by priority. Typical use case of priority queue is scheduling jobs. Each job has a priority and we process jobs in order of decreasing priority. While the current job is processed and new jobs may arrive.
Priority Queue has some main operations:
Insert(p): Adds a new element with priority p.
ExtractMax(): Extracts an elements with maximum priority.
ChangePriority(it, p): Changes the priority of an element pointed by it to p.
Priority Queue is used in many algorithms:
Dijkstra’s algorithm: finding a shortest path in a graph.
Prim’s algorithms: constructing a minimum spanning tree of a graph.
Huffman’s algorithm: constructing an optimum prefix-free encoding of a string.
Heap sort: sorting a given sequence.
You can implement priority with unsorted/sorted Array or List but each one has an trade off:
But by using binary heap, we can do Insert with O(logn) and ExtractMax with O(logn).
Binary heap has 2 types: binary min-heap and binary max-heap. In this article, we will discuss about binary max-heap. On the other side, binary min-heap has the same way of implementation.
Binary max-heap is a binary tree(each node has zero, one or two children) where the value of each node is at least the values of its children.
As you see, every children node has value not larger than the parent node. The root node has biggest value.
Max-Heap has some basic operations:
GetMax: just return the root value. Cost O(1).
Insert: We will attach a new node to any leaf. If it violate the heap property, we will bubble up(called SiftUp) new node until the heap property is satisfied. O(tree height)
As you see, we attach new node 32 to node 7, and we bubble it to node 29.
ExtractMax: Root node R is the max value of max-heap. To pop out the root node, we just swap root node R to any leaf node A and remove node R. It may violate the property of heap. We will do SiftDown operation. As a parent node A, we want to SiftDown to children B and C, we will choose the max value of children B and C, and swap it to node A. We SiftDown until the property of heap is satisfied.
We swap root node 42 and random leaf node 12 first, and remove node 42. The max-heap is violated now, we need to sift down node 12 as root down to children. At the final step, we have node 29 is root node.
ChangePriority: We need to change priority of a node. Depend on the value of new priority we will do SiftDown or SiftUp to qualify the heap property.
Remove: To remove a node A. We do it by 2 steps:
Step 1: Make that value of node A becomes biggest(infinity). We will do SiftUp operation to make node A become a root node.
Step 2: We do ExtractMax operation that describes above.
We know how heap works, we need to find out the way to build a heap. To build a heap and keep a binary heap-tree shallows, we have to fill the new node from left to right on last level. It’s called complete binary tree. Formally:
A binary tree is complete if all its levels are filled except possibly the last one which is filled from left to right. So the height of a complete binary tree with n nodes is at most O(logn).
A complete binary tree has elements filled from left to right, so we can store the tree as an Array where
Because we store as an array, each operation we perform need to keep the tree completed.
# we store elements in array Hdef parent(i):  return i/2def left_child(i):  return 2idef right_child(i):  return 2i + 1def sift_up(i):  while i > 1 and H[parent(i)] < H[i]:    swap H[parent(i)] and H[i]    i <- parent(i)def sift_down(i):  maxIndex <- i  left <- left_child(i)  if left <= size and H[left] > H[maxIndex]:    maxIndex <- left  right <- right_child(i)  if right <= size and H[right] > H[maxIndex]:    maxIndex <- right  if i != maxIndex:    swap H[i] and H[maxIndex]    SiftDown(maxIndex)def insert(p):  if size = maxSize:    return ERROR  size <- size + 1  H[size] <- p  SiftUp(size)def extract_max():  result <- H[1]  H[1] <- H[size]  size <- size - 1  SiftDown(1)  return resultdef remove(i):  H[i] = ∞  sift_up(i)  extract_max()def change_priority(i, p):  oldp <- H[i]  H[i] <- p  if p > oldp:    sift_up(i)  else:    sift_down(i)
Back to sorting problem, we observer that the root node of a max heap is the maximum value. What if we do ExtractMax() and put into another array, the result will be sorted decrease array. It’s fundamental of selection sort. It costs O(nlogn).
def heap_sort_selection_sort(A[1...n]):  create an empty priority queue  for i from 1 to n:    Insert(A[i]) # Insert operation to build max-heap.  for i from n downto 1:    A[i] <- ExtractMax()
In the matter of fact, we can do better heap-sort algorithm that didn’t use any extra array, it’s in-place heap sort algorithm. Given an array
A = [ 4, 1, 3, 2, 16, 9, 10, 14 , 8, 7]
We represent this array A as a heap:
Now we need to repair all nodes to satisfy the property of max-heap. We can start repairing the nodes in all subtrees of depth 1(depth 0 is all the leaves). The node from depth 1 is from n/2 down to 1
def build_heap(A[1...n]):  size <- n  for i from n/2 downto 1:    SiftDown(i)
We start from parent node 5, 4, 3, 2, 1([16, 2, 3, 1, 4]). Each step, we do SiftDown operation. Finally, we can have a max heap(f).
To sort a max heap, we just swap the node A[1] and A[size], remove A[size] and SiftDown(1).
def heap_sort(A[1...n]):  repeat (n-1) times:    swap A[1] and A[size]    size <- size - 1    SiftDown(1)
Given an array A[1...n], and 1 ≤ k ≤ n. Output the last k elements of a sorted version of A.
With heap sort, we easily solve this problem:
def partial_sorting(A[1...n], k):  build_heap(A)  for i from 1 to k:    extract_max()
The running time is: O(n+klogn). For the small number k = O(n/logn), the cost will be O(n). Really impressive.
Disjoint sets has many applications, one of them is determining the connected components of an undirected graph.
Disjoint set data structure maintains the collection of S1, S2, ..., Sk of disjoint dynamic set. Each set we represent as a rooted tree.
We can store these set(rooted trees) by using an array which is the value A[i] is the parent of node i or node i if it is the root.
Disjoint set has some main operations:
MakeSet(i): Create a set contain only one i: {i}.
def make_set(i):  parent[i] <- i
Find(i): Finding i in a set, return root index
def find(i):  while i != parent[i]:    i <- parent[i]  return i
Union(S1, S2): Merge 2 rooted trees. It’s the most important operation of disjoint set.
We have 2 rooted trees and we want to union these trees. Normally, we have 2 ways to do that:
The good one is the one with shorter of height. How do we know to choose the root node for union set?
To answer that question, we just need to use a rank as heigh to choose the final root. In other words, we just hang the shorter one under the root of a taller one. To avoid re-calculating the rank(height) each time to do union operation, we need to store height of each subtree in an array rank[1...n] where rank[i] is the height of the subtree whose root is i.
We change our implements of disjoint set operations:
def make_set(i):  parent[i] <- i  rank[i] <- 0def find(i):  while i != parent[i]:    i <- parent[i]  return idef union(i, j):  i_id <- find(i)  j_id <- find(j)  if i_id == j_id:    return  if rank[i_id] > rank[j_id]:    parent[j_id] <- i_id  else:    parent[i_id] <- j_id    if rank[i_id] == rank[j_id]:      rank[j_id] <- rank[j_id] + 1
Let visualize the process of building a disjoint set, and do union(2, 4), union(5, 2), union(3, 1), union(2, 3), union(2,6).
Each step, we need to choose lower rank tree and attach it to higher rank tree. We only increase rank of result when we have 2 trees with the same rank.
While building the tree, we can immediately realize that our tree will become higher, higher. Let see how to find element 6, we do travel to all parent node to get root node 5.
Notice that all node 6, 12, 3 has same root node 5, so it will be wonderful if we can transform all their parent node into direct parent node 5 likes this one:
You can see, we can compress the height and we can find element faster. You will be surprise to see that the implement of this heuristic so simple:
def find(i):  if i != parent[i]:    parent[i] <- find(parent[i])  return parent[i]
In this problem you will convert an array of integers into a heap. This is the crucial step of the sorting algorithm called HeapSort. It has guaranteed worst-case running time of O(n log n) as opposed to QuickSort’s average running time of O(n log n). QuickSort is usually used in practice, because typically it is faster, but HeapSort is used for external sort when you need to sort huge files that don’t t into memory of your computer.
Problem Description
Task. The first step of the HeapSort algorithm is to create a heap from the array you want to sort. By the way, did you know that algorithms based on Heaps are widely used for external sort, when you need to sort huge files that don’t t into memory of a computer?
Your task is to implement this first step and convert a given array of integers into a heap. You will do that by applying a certain number of swaps to the array. Swap is an operation which exchanges elements ai and aj of the array a for some i and j. You will need to convert the array into a heap using only O(n) swaps, as was described in the lectures. Note that you will need to use a min-heap instead of a max-heap in this problem.
Input Format. The first line of the input contains single integer n. The next line contains n space-separated integers ai.
Output Format. The first line of the output should contain single integer m — the total number of swaps. m must satisfy conditions 0 ≤ m ≤ 4n. The next m lines should contain the swap operations used to convert the array a into a heap. Each swap is described by a pair of integers i,j — the 0-based indices of the elements to be swapped. After applying all the swaps in the specified order the array must become a heap, that is, for each i where 0 ≤ i ≤ n − 1 the following conditions must be true:
1. If 2i+1 ≤ n−1,then ai < a(2i+1).2. If 2i+2 ≤ n−1,then ai < a(2i+2).
Note that all the elements of the input array are distinct. Note that any sequence of swaps that has length at most 4n and after which your initial array becomes a correct heap will be graded as correct.
Sample 1.
Input:
55 4 3 2 1
Output:
31 40 11 3
Explanation:
After swapping elements 4 in position 1 and 1 in position 4 the array becomes 5 1 3 2 4.After swapping elements 5 in position 0 and 1 in position 1 the array becomes 1 5 3 2 4.After swapping elements 5 in position 1 and 2 in position 3 the array becomes 1 2 3 5 4, which is already a heap, because a0 =1 < 2 = a1, a0 =1 < 3=a2, a1=2 < 5=a3, a1=2 < 4=a4.
Change the BuildHeap algorithm from the lecture to account for min-heap instead of max-heap and for 0-based indexing.
While building a heap, we will do SiftDown operation from n/2-th down to 1-th node to repair a heap to satisfy min-heap property. Each time we swap nodes, we need to track it .
In this problem you will simulate a program that processes a list of jobs in parallel. Operating systems such as Linux, MacOS or Windows all have special programs in them called schedulers which do exactly this with the programs on your computer.
Problem Description
Task. You have a program which is parallelized and uses n independent threads to process the given list of m jobs. Threads take jobs in the order they are given in the input. If there is a free thread, it immediately takes the next job from the list. If a thread has started processing a job, it doesn’t interrupt or stop until it finishes processing the job. If several threads try to take jobs from the list simultaneously, the thread with smaller index takes the job. For each job you know exactly how long will it take any thread to process this job, and this time is the same for all the threads. You need to determine for each job which thread will process it and when will it start processing.
Input Format. The first line of the input contains integers n and m.
The second line contains m integers ti — the times in seconds it takes any thread to process i-th job. The times are given in the same order as they are in the list from which threads take jobs.
Threads are indexed starting from 0.
Output Format. Output exactly m lines. i-th line (0-based index is used) should contain two space- separated integers — the 0-based index of the thread which will process the i-th job and the time in seconds when it will start processing that job.
Sample 1.
Input:
2 5 1 2 3 4 5
Output:
0 01 00 11 20 4
Explanation:
1. The two threads try to simultaneously take jobs from the list, so thread with index 0 actually takes the first job and starts working on it at the moment 0.2. The thread with index 1 takes the second job and starts working on it also at the moment 0.3. After 1 second, thread 0 is done with the first job and takes the third job from the list, and starts processing it immediately at time 1.4. One second later, thread 1 is done with the second job and takes the fourth job from the list, and starts processing it immediately at time 2.5. Finally, after 2 more seconds, thread 0 is done with the third job and takes the fifth job from the list, and starts processing it immediately at time 4.
Think about the sequence of events when one of the threads becomes free (at the start and later after completing some job). How to apply priority queue to simulate processing of these events in the required order? Remember to consider the case when several threads become free simultaneously.
Beware of integer over ow in this problem: use type long long in C++ and type long in Java wherever the regular type int can over ow given the restrictions in the problem statement.
The idea is we need a priority queue to keep workers. We compare 2 workers by its status and index. To choose the next worker, we just choose the root min-heap tree. To mark it working, we need to ChangePriority of root tree and repair tree.
In this problem, your goal is to simulate a sequence of merge operations with tables in a database.
Problem Description
Task. There are n tables stored in some database. The tables are numbered from 1 to n. All tables share the same set of columns. Each table contains either several rows with real data or a symbolic link to another table. Initially, all tables contain data, and i-th table has ri rows. You need to perform m of the following operations:
1. Consider table number destination(i). Traverse the path of symbolic links to get to the data. That is, while destination(i) contains a symbolic link instead of real data do destination(i) ← symlink(destination(i))
2. Consider the table number source(i) and traverse the path of symbolic links from it in the same manner as for destination(i).
3. Now, destinationi and source(i) are the numbers of two tables with real data. If destination(i) != source(i), copy all the rows from table source(i) to table destination(i), then clear the table source(i) and instead of real data put a symbolic link to destination(i) into it.
4. Print the maximum size among all n tables (recall that size is the number of rows in the table). If the table contains only a symbolic link, its size is considered to be 0.
See examples and explanations for further clarifications.
Input Format. The first line of the input contains two integers n and m — the number of tables in the database and the number of merge queries to perform, respectively.
The second line of the input contains n integers ri — the number of rows in the i-th table.
Then follow m lines describing merge queries. Each of them contains two integers destination(i) and source(i) — the numbers of the tables to merge.
Output Format. For each query print a line containing a single integer — the maximum of the sizes of all tables (in terms of the number of rows) after the corresponding operation.
Sample 1.
Input:
5 5 1 1 1 1 13 52 41 45 45 3
Output:
22355
Explanation:
In this sample, all the tables initially have exactly 1 row of data. Consider the merging operations:
1. All the data from the table 5 is copied to table number 3. Table 5 now contains only a symbolic link to table 3, while table 3 has 2 rows. 2 becomes the new maximum size.
2. 2 and 4 are merged in the same way as 3 and 5.
3. We are trying to merge 1 and 4, but 4 has a symbolic link pointing to 2, so we actually copy all the data from the table number 2 to the table number 1, clear the table number 2 and put a symbolic link to the table number 1 in it. Table 1 now has 3 rows of data, and 3 becomes the new maximum size.
4. Traversing the path of symbolic links from 4 we have 4→2→1, and the path from 5 is 5→3. So we are actually merging tables 3 and 1. We copy all the rows from the table number 1 into the table number 3, and now the table number 3 has 5 rows of data, which is the new maximum.
5. All tables now directly or indirectly point to table 3, so all other merges won’t change anything.
Sample 2.
Input:
6 410 0 5 0 3 3 6 66 55 64 3
Output:
10 10 10 11
Explanation:
In this example tables have diferent sizes. Let us consider the operations:
1. Merging the table number 6 with itself doesn’t change anything, and the maximum size is 10 (table number 1).
2. After merging the table number 5 into the table number 6, the table number 5 is cleared and has size 0, while the table number 6 has size 6. Still, the maximum size is 10.
3. By merging the table number 4 into the table number 5, we actually merge the table number 4 into the table number 6 (table 5 now contains just a symbolic link to table 6), so the table number 4 is cleared and has size 0, while the table number 6 has size 6. Still, the maximum size is 10.
4. By merging the table number 3 into the table number 4, we actually merge the table number 3 into the table number 6 (table 4 now contains just a symbolic link to table 6), so the table number 3 is cleared and has size 0, while the table number 6 has size 11, which is the new maximum size.
Think how to use disjoint set union with path compression and union by rank heuristics to solve this problem. In particular, you should separate in your thinking the data structure that performs union/find operations from the merges of tables. If you’re asked to merge first table into second, but the rank of the second table is smaller than the rank of the first table, you can ignore the requested order while merging in the Disjoint Set Union data structure and join the node corresponding to the second table to the node corresponding to the first table instead in you Disjoint Set Union. However, you will need to store the number of the actual second table to which you were requested to merge the first table in the parent node of the corresponding Disjoint Set, and you will need an additional eld in the nodes of Disjoint Set Union to store it.
We use parent array to store disjoint set.
We use lines array to store maximum of root i.
We need to build disjoint set with raking and path compression heuristic.
See the chapter 6.4 in [CLRS] — Priority Queue
See this min-heap visualization.
See section 5.1.4 in [DPVo8] — Disjoint set
Also see this tutorial on Disjoint Sets data structures.
Also see this visualization of Disjoint Sets with and without Path Compression and Union by Rank heuristics.
[DPV08] Sanjoy Dasgupta, Christos Papadimitriou, and Umesh Vazirani. Algorithms (1st Edition). McGraw-Hill Higher Education. 2008.
[CLRS] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein. Introduction to Algorithms (3rd Edition). MIT Press and McGraw-Hill. 2009.