Lecture 5--Recurrence relations; generating functions. 10/3/97

================================================================================

Section 5.1. Recurrence relations.

Many of our analyses of algorithm performance wlll require us to compute the "closed form" of a recurrence relation. This can often be done by noticing a pattern, which can be verified by induction. A more formal way to compute the closed for is to use generating functions.

Example. Suppose T is defined on the positive integers by

T(1) = a

T(n) = T(|_(n/2)_|) + a , n >= 1.

T is an example of a recurrence relation, i.e., the value of T for a specific argument n is defined in terms of values of T for arguments < n. (The Fibonacci numbers referred to in Lecture 4 are also defined through a recurrence relation.)

We will show two ways of computing a "closed form" expression for T:

Method 1. "Guess and prove".

Example. Suppose T(1) = a,

T(n) = T( |_n / 2 _| ) + a , n > 1, where a > 0 is a constant.

Now by substitution T(n) = T( |_n/2_| ) + a = T( |_n/4_| ) + a + a = .....

= T(1) + a + a + . . . + a .

How many copies of a are in the last sum? If n = 2^k for some k, then we get to the

last step after about log₂n steps. So guess that

T(n) = a|_ log₂n _| + a.

We can prove that our guess is correct (for all positive n) by induction on n as follows:

Step 1. The equation holds for n = 1.

Step 2. Assume that for 1 <= k < n the equation holds (using a slight variation of step 2 in the induction section in lecture 4).

Then T(n) = a + T( |_n/2_|) by the recurrence relation

= a + a( |_ log₂ (n/2) _| ) + a by the induction hypothesis

= 2a + a( |_ log₂ (n/2) _| ) .

Now we have 2 cases to consider.

If n is odd, |_ n/2_| = (n-1) / 2. So T(n) = 2a + a(|_ log₂ (n-1) - 1 _| )

= a + a |_ log₂(n-1) _| = a + a |_ log₂n _| since n is odd.

So we have successfully completed step 2 of the induction proof.

Step 3. Since we have successfully completed steps 1 and 2 we conclude that the desired equality holds.

Another example. If T(n) = 2T( |_ n/2 _| ) + n-1 for n >= 1, T(1) = 0, then

T(n) <:= cn|_log₂n_| for some constant c.

Exercise 5.1 (grad). Prove that the identity in the previous line is valid.

Exercise 5.2. If T(n) = 3T( |_ n/4 _| ) + n-1 for n >= 1, T(1) = 1, use the guessing method to find a closed form expression for T(n).

5.2. Generating functions. A mathematically interesting method for finding the closed form expression for a recurrence relation is to use the idea of a generating function. A generating function is an example of a transform. We transform the function T(n) defined on the positive integers to a related function g(x) defined for some nonempty set of real numbers x. There is a 1-1 relationship between T(n) and g(x), i.e., a given T always transfroms to the same g(x) and vice versa. This is the same idea we use, for example, to replace a distribution function by its laplace transform or its fourier transform.

Formally we define the transform g(x) of T(n) by

g(x) = "SUM"_0<=n<=ooT(n)xⁿ.

Example. If T(n) = 2T(n-1) - 1, N >= 1, T(0) = 0, then the generating function for T is

g(x) = x + 3x² + 7x³ + ......

Now recall the trick used to find the sum of a geometric series with ratio r. We compute the partial sums S_N and rS_N and subtract, then let N -> 00 to find the limit. We use the same idea here, but instead of a ratio r we must multiply by an expression derived from the recurrence relation. In this case T(n) = 2T(n-1) implies we need to shift by 1 ( n -> n-1) and multiply by 2. If we do this we get

g(x) - 2xg(x) = x + x² + x³ + . . . = 1 / (1-x) for 0 <= x < 1.

Factoring out g(x) from the lhs and doing some algebra gives

g(x) = 1 / (1-x)(1-2x) = 1 / (1-2x) - 1 / (1 -x).

Now we can rewrite the two terms on the rhs as geometric series with rations 2x and x respectively, to get

g(x) = "SUM"_0<=n<=oo(2x)ⁿ - "SUM"_0<=n<=ooxⁿ

= "SUM"_0<=n<=oo(2ⁿ - 1)xⁿ

Since the generating function of a recurrence relation is unique, we have

g(x) = "SUM_"0<=n<=ooT(n)xⁿ = "SUM"_0<=n<=oo(2ⁿ - 1)xⁿ, and so

T(n) = 2ⁿ - 1.