Class 09: Analyzing Algorithms

Held Monday, February 7, 2000

Overview

Today we consider ways in which one might choose between different algorithms that purport to solve the same problem. We look at a mathematical notation, Big O, that lets us describe the space or time requirements of an algorithm.

Question 9 for today: Suppose you've been given two sets of instructions that solve the same problem. What criteria might you use to determine whether one is better than the other?

Question 10 for Tuesday: Describe how to put a pile of books in alphabetical order by author.

Notes

• No readings for tomorrow! Wo hoo!
• Some of you may be wondering about how you're doing. I hope to give you preliminary evaluations at the end of the week.

Contents

• Three Smallest Algorithms
  • Choosing Between The Algorithms
• Evaluating Algorithms
  • Big O Notation
• Searching

Summary

• Comparing various ``smallest'' algorithms
• Evaluating algorithms
• Formal analysis: Big O notation
• Searching
• Handouts:
  • Your answers to question 9: Suppose you've been given two sets of instructions that solve the same problem. What criteria might you use to determine whether one is better than the other?

Three Smallest Algorithms

• On Friday, we saw two algorithms for finding the smallest value in a list of numbers. Here are those two, plus four more.
• Algorithm A:

  1. Put the list in order
  2. Return the first element
  

• Algorithm B:

  1. Set guess to the first element of the list
  2. For each element, e, of the list
     a. if (e < guess) then
        i. guess = e.
  3. Return guess
  

• Algorithm C:

  1. Set guess to the last element of the list
  2. For each element, e, of the list
     a. if (e < guess) then
        i. guess = e.
  3. Return guess
  

• Algorithm D:

  1. Return the last element of the list
  

• Algorithm E:

  1. For each number, make a pile of Lego (tm) pieces stacked
     on top of each other.
  2. Put a board higher than all of the piles and move it down.
       a. Whenever it hits a pile, remove that pile
       b. Stop when one pile is left
  3. The number of Lego (tm) pieces is the smallest grade
  

• Algorithm F:

  1. If there is only one grade in the list,
       a. Return that element
     Otherwise
       b. Split the list into two equal halves
       c. Find x, the smallest grade in the first half
       d. Find y, the smallest grade in the second half
       e. Return the smaller of x and y
  

Choosing Between The Algorithms

• How do we choose between these algorithms?
• Which do you prefer and why?

Evaluating Algorithms

• As many of you suggested in your answers to today's question, there are a number of criteria we might use when evaluating algorithms.
  • We'll assume that all the algorithms are unambiguous
• Clarity: How easy is it to understand the algorithm?
• Simplicity: How simple are the steps? (How would we evaluate this formally?)
• Precision: Does it give the correct answer?
• Length: How short is it?
• Speed: How fast is it?
• Note that there are not always ways to choose one algorithm over another. Two algorithms may be essentially equivalent (as in algorithms B and C above)
• There are also some other issues we might consider.
• Generality: Can the same algorithm solve more than one problem?
• Modifiability: How easy is it to change the algorithm to solve other problems?
• Resource Use: How much ``stuff'' does the algorithm use?

Big O Notation

• Unfortunately, it is difficult to precisely analyze how long an algorithm takes.
• Different ``steps'' can take different amounts of time.
• The same algorithm can take different amounts of time depending on the input.
  • In algorithm B above, in some lists of grades we will update guess many times; in others, we won't update it much.
• We need a way to describe the running time without worrying about these issues.
  • Since the running time is likely to depend on the ``size'' of the problem, we represent that as a parameter, n
• For this purpose, we use Big-O notation.
  • We say that an algorithm takes O(n) steps if it never takes more than c*n steps, for some constant n.
  • Similarly, we say that an algorithm takes O(n²) steps if it never takes more than c*n*n steps, for some constant c.
• We'll look at a few sample algorithms over the next few days.

Searching

• If there is sufficient time, we will consider mechanisms for searching for a value in an array.