Lecture 13. Heapsort. 10/23/97.

================================================================

NOTE: CORRECTION FOR PROGRAMMING ASSIGNMENT.

REPLACE THE PROCEDURE buildheap(m,endofheap) with the following:

procedure buildheap (k)

        for j = k downto 0 do adjustheap(j,k)

13.1. Description of heapsort. In the previous lectures we have seen sorting algorithms with worst case behavior "capital theta" (n²) and one sorting algorithm, quicksort, with average behavior "capital theta"(nlog₂n). Later we will prove that any sorting algorithm which sorts an array of n keys using comparisons must make AT LEAST "capital omega" (nlog₂n) comparisons on some input data and therefore will always run in time

"capital omega" (nlog₂n).

In this lecture we will look at the algorithm heapsort which ALWAYS sorts its input data in time "capital theta"(nlog₂n). Heapsort also sorts the data "in place", without requiring an auxiliary array. Therefore, theoretically at least, heapsort is the "best" sorting algorithm we have seen so far. In practice, however, it turns out that the number of steps heapsort does in each iteration is often much larger that the number of steps quicksort does in each iteration, and so generally in practice quicksort will have better performance. Heapsort is interesting, nevertheless, because its running time is theoretically the best we can ever get for a sorting algorithm which sorts by comparisons. Heapsort is also of independent interest because it is based on the heap, a useful data structure for maintaining a priority queue, for example.

Definition. A (binary) heap is a complete binary tree in which the key stored at a node is always >= the key stored at each of its children (recall that a complete binary tree is one in which every level l but the last is full, i.e., contains 2^l nodes, and all the nodes in the last level are as far to the left as possible). A heap can be efficiently implemented in an array by noticing that the children of node j can (in c or c++) be placed at positions 2j + 1 and 2j + 2.

Example:

the heap 8 / \ / \ 4 6 / \ / \ / \ 5 2 0 3 / \ / -1 -4 -2

corresponds to the array position: 0 1 2 3 4 5 6 7 8 9 value: 8 4 6 0 3 5 2 -1 -4 -2

The heap is a very useful data structure in its own right. For example, a heap can be used to implement a priority queue efficiently because the two tasks of removing the maximum element from a heap and inserting an element into a heap can each be carried out in "capital theta"(log₂n) time.

The heapsort procedure is fairly straightforward. Given an array L with n elements, we first rearrange the elements in L so that L becomes a heap, with the children of node j at positions 2j + 1 and 2j + 2. Now the largest element of L is at position 0. The sort now proceeds as follows, placing the elements from largest to smallest in positions n-1, n-2, . . . , 0 :

      for i = n-1 downto 1

          swap L[0], the largest element in the still unsorted part
          of the list, with L[i]

          rearrange the elements in positions L[0], ... , L[i-1] so that
          they form a heap (with the next largest element now at L[0] )

First we show by induction: if the items at j+1,j+2, . . . , n-1 are all the roots of heaps then after adjustheap(j,n-1) all of the items at j,j+1,j+2, ... , n-1 will be roots of heaps.

Step 1. For j = n-1, adjustheap(n-1,n-1) does nothing, so the statement is trivially true.

Step 2. If j is a leaf or has no child with a larger element, then the statement is trivially true. If j has one child (2j + 1) and L[2j+1] < L[j], then we swap L[j] with L[2j+1] and call adjustheap(2j+1,n-1). By the induction hypothesis, the tree with root 2j+1 is made into a heap, and vertices j+1, j+2, ..., 2j were already the roots of heaps. After the swap, L[j] > L[2j+1], so the tree with root j is also a heap.

Step 3. Since steps 1 and 2 hold, we have proved that all the items at j, j+1, j+2, ..., are roots of heaps.

Next we show by induction that heapsort will correctly sort the items in L:

Proof.

Step 1. If n = 2, then first we arrange the heap, with L[0] the largest item, and then we swap L[0] and L[1], so heapsort works correctly in this case.

Step 2. Assume that heapsort correctly sorts arrays of k items, for k < n. Assume L is a list of n items. Then heapsort will first rearrange L to be a heap and then swap L[0], the largest item, with L[n-1]. Now the largest item is in the last position, where it belongs, and there are only n-1 items remaining to be sorted. By induction heapsort will sort these n-1 items correctly, so the entire list will be sorted correctly.

Step 3. Since we have verified steps 1 and 2, we have shown by induction that heapsort is a correct algorithm for sorting an array of keys.

Example: suppose L contains 10 elements:

array position: 0 1 2 3 4 5 6 7 8 9

stored value: 2 8 4 3 -2 6 5 0 1 -1

actions: a -1 -2

b 6 4

c 8 2

d 3 2

Steps in heapsort(L):

    buildheap (0,9):
        for j = 9,8,7,6,5 adjustheap (j,9) does nothing, since these
            are all leaves
        adjustheap(4,9) -2 < -1 so swap and adjustheap(9,9)--no
              further action (a)
        adjustheap(3,9): nothing to do, 0 and 1 both < 3 
        adjustheap(2,9): swap 6 and 4; then no further action needed (b)
        adjustheap(1,9):  nothing to do, 3 and -2 are both <8
        adjustheap(0,9):  swap 2 with 8 (c)
                  adjustheap(1,9): swap 2 and 3; now nothing more to do

So now we have the heap

array position:0 1 2 3 4 5 6 7 8 9

stored value: 8 3 6 2 -1 4 5 0 1 -2

Now loop from 9 down to 1:

swap 8 with -2, adjustheap(0,8): 6 3 5 2 -1 4 -2 0 1 8

swap 6 with 1, adjustheap(0,7): 5 3 4 2 -1 1 -2 0 6 8

swap 5 with 0, adjustheap(0,6): 4 3 0 2 -1 1 -2 5 6 8

swap 4 with -2, adjustheap(0,5): 3 2 0 -2 -1 1 4 5 6 8

swap 3 with 1, adjustheap(0,4): 2 1 0 -2 -1 3 4 5 6 8

swap 2 with -1, adjustheap(0,3): 1 -1 0 -2 2 3 4 5 6 8

swap -2 with 1, adjustheap(0,2): 0 -1 -2 1 2 3 4 5 6 8

swap 0 with -2, adjustheap(0,1): -1 -2 0 1 2 3 4 5 6 8

swap -1 with -2, adjustheap (0,0) -2 -1 0 1 2 3 4 5 6 8

Exercise 13.1. Show the steps in heapsort to sort the 12-element array L which initially has contents 5,8,3,2,0,1,9,4,6,7,11,-3

13.2. Calculating the (worst case) running time for heapsort. In order to calculate the worst case running time of heapsort, we first calculate the running times of the procedures adjustheap and buildheap.

Time taken by adjustheap (m, end_of_heap): the time taken by this procedure satisfies the recurrence relation

T(h) <= T(h-1) + c , T(0) = d,

where h is the height of m and c and d are constants. It is easy to calculate that

T(h) <= hc + d,

i.e., T = O(h).

Exercise 13.2 Give algorithms to remove the maximum element from a heap (without destroying the heap property) and to insert an element in a heap. What are the worst case running times of your algorithms?

Exercise 13.3. (grad). Show that in a heap of n elements at most ceiling(n / 2^j+1) nodes are at height j.

Time taken to build a heap: buildheap calls adjustheap once for each node. Thus the time it takes to run is on the order of the sum, over all nodes, of the heights of these nodes. Using exercise 13.2 we have that the time to heapsort an array of n elements, T(n), satisfies

T(n) <= "SUM"_0<=i<=logn(i*(n / 2ⁱ ))

= O(n)

since

"SUM"_0<=i<=logn(i* (1 / 2ⁱ )) = "capital theta" ("integral" _1<=i<=lognx2^-x ) = O (1).