# Class 20: More Algorithm Analysis

Back to Algorithm Analysis. On to Tail Recursion.

Held: Friday, 21 February 2003

Summary: Today we continue with our investigation of algorithm analysis.

Related Pages:

Due

Assignments:

Notes:

• I'll be gone from about noon today until Sunday evening. I'll be serving on the steering committee of the EdMedia World Conference of Educational Multimedia, Hypermedia, and Telecommunications.

Overview:

• Big O notation, revisited.
• Doing Big-O Analysis.
• Dominant terms.
• Big-O Analysis of Recursive Procedures.
• Experimental analysis.

## Review

• f(n) is in O(g(n)) iff
• there exists a number n0
• there exists a number d > 0
• for all n > n0, abs(f(n)) <= abs(d*g(n))
• This notation provides a convenient mechanism for talking about running time without worrying about too many potentially useless details.

## Eliminating Constants

• One of the nice things about asymptotic analysis is that it makes constants unimportant because they can be hidden in the d.
• If f(n) is 100n seconds and g(n) is 0.5n seconds, then
• f(n) is in O(g(n)) [let d be 200]
• g(n) is in f(n) [let d be 1]
• If f(n) is 100n seconds and g(n) is n2 seconds, then f(n) is in O(g(n)) [let n0 be 100 and d be 1]
• However, g(n) is not in O(f(n)). Why not?
• Suppose there were an n0 and a d.
• Consider what happens for n = 101d.
• d*f(n) = d*100*101*d = d*d*100*101.
• However, g(n) = d*d*101*101, which is even larger.
• If n0 is greater than 101d, we'll still have this problem [proof left to reader].
• Since constants can be eliminated, we normally don't write them.
• That is, we say that the running time of an algorithm is O(n) or O(n2) or ....

## Doing Big-O Analysis

• Most computer scientists begin their study of Big-O by analyzing simple iterative functions.
• Here's how you start:
• Count each step as one.
• Count each conditional as taking one plus the maximum of the number of steps in each branch.
• Count each loop as taking the number of repetitions times the cost of the body (perhaps plus one).
• Count each procedure call as taking one plus the number of steps in the procedure call.
• You can then simplify the result by dropping lower-order terms and eliminating constants.

## Relating Functions

• Note that some of the counting can be hard, so we may also need to estimate throughout the process.
• We can often express the running time of an algorithm as a composition of other functions.
• For example, we might have a primary control structure which repeats its contents O(h(n)) times and within that control structure, we have a subalgorithm that takes O(g(n)) time.
• Similarly, we might break our algorithm up into two independent algorithms, one of which takes O(h(n)) time and one of which takes O(g(n)) time.
• We might also look at other relationships.
• Here are some questions we might ask ourselves about composition of functions.
• If f(n) is in O(h(n)) and g(n) is O(k(n)), then what can we say about f(n) + g(n)?
• If f(n) is in O(h(n)) and g(n) is O(k(n)), then what can we say about f(n) * g(n)?
• If f(n) is in O(g(n)) and g(n) is O(h(n)) then what can we say about f(n) and h(n)?
• If f(n) is in O(kg(n)) and k is a constant, then what else can we say about f(n) and g(n)?
• If f(n) is in O(g(n) + h(n)) and g(n) is in O(h(n)), then what simpler thing can we say about f(n)?

## Analyzing Recursive Procedures

• Unfortunately, the techniques for analyzing algorithms we discussed above focuse primarily on iterative algorithms. What about recursive algorithms?
• The process essms inherently more difficult.
• However, we can use a similar strategy.
• Count the number of recursive calls.
• Count the time spent during each recursive call (except for the time spent in subsequent recursive calls).
• How do you count the number of recursive calls? Often, the shrink function gives you a sense of how many recursive calls you need to do
• That is, how many calls to shrink does it take to reduce the argument to the base case.
• It is also possible to use the concept of recurrence relations to determine the running time of an algorithm.
• For example, if a recursive algorithm takes c steps and reduces the parameter by one, we might express the running time as a function, time(n) which represents the running time on input of n elements:
• time(n) = c + time(n-1)
• time(1) = c
• This equation is relatively easy to solve "intuitively".
• time(n) = c + time(n-1)
• = c + c + time(n-2)
• = ...
• = c + c + ... + time(1)
• = c + c + ... + c [n times]
• = c*n
• Other interesting recurrence relations:
• time(n) = n + time(n-1)
• time(n) = c n + time(n-1)
• time(n) = c + time(n/2)
• time(n) = n + time(n/2)
• time(n) = c * time(n-1)
• time(n) = c * time(n/2)

## Experimental Analysis

• Particularly as you start to analyze algorithms, it may be helpful to analyze them experimentally as well as abstractly.
• What do I mean when I say analyze them experimentally? I mean that we can time them on a variety of inputs and graph the results.
• If the experimental and the abstract results match, we can be fairly confident in the abstract results.
• If they don't, we may need to try again.
• Note that some analyses can be very difficult.

## History

Thursday, 15 January 2003 [Samuel A. Rebelsky]

• Created as a mostly-blank outline.

Friday, 21 February 2003 [Samuel A. Rebelsky]

• Filled in the details.
• Many of those details were taken from old sections of CSC152 and then rewritten and reformatted.

Back to Algorithm Analysis. On to Tail Recursion.

Disclaimer: I usually create these pages on the fly, which means that I rarely proofread them and they may contain bad grammar and incorrect details. It also means that I tend to update them regularly (see the history for more details). Feel free to contact me with any suggestions for changes.

This document was generated by Siteweaver on Tue May 6 09:20:35 2003.
The source to the document was last modified on Fri Feb 21 09:34:01 2003.
This document may be found at `http://www.cs.grinnell.edu/~rebelsky/Courses/CS153/2003S/Outlines/outline.20.html`.

You may wish to validate this document's HTML ; ; Check with Bobby

Samuel A. Rebelsky, rebelsky@grinnell.edu