Fundamentals of Computer Science II (CSC-152 2000S)


Notes on Assignment 3: Algorithm Analysis

Preliminaries

Do not read these notes until after you've turned in the assignment.

As you may have noted, I've released this answer key before grading your homeworks. That's because I think you'll want to refer to it as you work on the exam. I'll make some general comments on your assignments after I've graded them.

As many of you have noted, this is a lot like a long math assignment. Just as in math assignments you don't get all of the problems graded, you may find that I will not grade all of the problems on this assignment.

Here are the code files:

1. Big-O Computation

Bound the running time using Big-O notation of each of the following algorithms. Try to make the bound as close as possible. Note that you may have to come up with an appropriate metric for the size of each problem.

(a) Smallest Value

To find the smallest value in a collection ...
Set smallestSoFar to one value in the collection     O(1) steps
While unchecked values remain in the collection      O(n) repetitions
  Pick one                                             O(1) steps
  If that value is smaller than guess then             O(1) steps
    Set smallestSoFar to that value                    O(1) steps
Return smallestSoFar                                 O(1) steps

So, the overall running time is O(2 + 3*n) which is O(n).

(b) Smallest Value Using Divide and Conquer

We analyze this using recurrence relations. We'll let f(n) represent the running time on input of size n. Note that I've rewritten the algorithm slightly to clarify things.

To find the smallest value in a collection ...
If the collection is empty then
  Report an error
Otherwise, if the collection has only one value then    f(1) = 
  Return that value                                       1
Otherwise                                               f(n) =
  Split the collection into two parts                     1 +
  Find the smallest value in the first part               f(n/2) +
  Find the smallest value in the second part              f(n/2) +
  Take the minimum of those two values                    1

First, we expand the recursive version to see if we see a pattern.

f(n) = 2 + 2*f(n/2)            // From above
     = 2 + 2*(2 + 2*f(n/4))    // Expanded f(n/2)
     = 2 + 4 + 4*f(n/4)        // Distributed 2*
     = 6 + 4*f(n/4)            // Added 2+4
     = 6 + 4*(2 + 2*f(n/8))    // Expanded f(n/4)
     = 6 + 8 + 8*f(n/8)        // Distributed 2*
     = 14 + 8*f(n/8)           // Added 6+8
     = 14 + 8*(2 + 2*f(n/16))  // Expanded f(n/8)
     = 14 + 16 + 16*f(n/16)    // Distributed 2*
     = 30 + 16*f(n/16)

Okay, I see a pattern. I note that

f(n) = 2*(2k - 1) + 2k*f(n/2k);

Now, I want the f term on the right to be f(1). This happens when

n/2k = 1
n = 2k    // Multiply both sides by denominator
k = log2n // Definition of log

Hence

f(n) = 2*(2log2n - 1) + 2log2n*f(1)
     = 2*(n - 1) + n*f(1)
     = 2n - 2 + n
     = 3n -2

Therefore, f(n) is in O(n).

Note that we could have done a simpler, but less formal analysis by noting that we look at each value at most twice (once when we return it in the singleton and once when we take the minimum of two values). Hence, this is O(n).

Anne and Amber also noted that we don't really need the log in this program. Once we decide that 2k is equal to n, we can just substitute everywhere. However, the technique of determining the value of k is useful for other problems, so I left the analysis as I originally did it.

(c) Least Difference

In the following, we'll let n be the number of values in the collection and p be the number of pairs.

To find the smallest difference between any two 
  different numbers in a collection ...
List all pairs of different numbers in the collection  // O(p)
Set estimate to                                        // O(1)
  the difference between the values in the first pair
For each remaining pair                                // O(p) repetitions
  Compute the difference between the two values          // O(1)
  If that differences is less than estimate then         // O(1)
    Set estimate to that difference                      // O(1)
Return estimate                                        // O(1)

Hence, the result is O(p + 1 + 3*p + 1) or simply O(p).

Now, what is p? If there are n values in the collection, then there are O(n2) pairs. Hence this is an O(n2) algorithm.

(d) Least Difference, Revisited

To find the smallest difference between any two 
  different numbers in a collection ...
Set estimate to "infinity"               // O(1)
For each value, v, in the collection     // O(n) repetitions
  For each value, u, not equal to v         // O(n) repetitions
    If |u-v| < estimate then                   // O(1) steps
      Set estimate to |u-v|                    // O(1) steps
Return estimate                          // O(1)

Hence, the running time is O(1 + n*n*2 + 1) which is O(n2).

2. Computing Square Roots with Divide and Conquer

Here's an iterative algorithm that finds the square root of a value to a specified accuracy.

/**
 * Compute the square root of val to accuracy 1/n.
 * Pre: (1) val >= 0
 *      (2) n >= 1
 *      (3) The square root of val can be represented.
 * Post: (1) Returns a value v such that |v-sqrt(val)| < 1/n.
 */
public static double sqrt(double val, long n) {
  // Compute 1/nth of the range of possible values
  double accuracy = 1.0/((double) n);
  // Start with an initial guess
  double guess = 0;
  // Determine the difference between the square of the guess
  // and the actual square root.
  double diffsquared = val;

  // Step through all the possible values between 0 and val.
  for (double nextguess = 0.0;
       nextguess <= val;
       nextguess = nextguess + accuracy) {
    // Compute the difference between the square of the guess
    // and the actual value.
    double nextdiff = Math.abs(nextguess*nextguess-val);
    // If it's better, use it.
    if (nextdiff < diffsquared) {
      guess = nextguess;
      diffsquared = nextdiff;
    }
  } // for
  return guess;
} // sqrt(double,double)

As you might be able to tell, what this algorithm basically does is divide the interval between 0 and val into potential guesses that occur every 1/n units. It then tries each of these guesses and uses the best. Why 0 and val? Because we know that the square root of val is at least 0 and no more than val.

The running time of this algorithm is somewhat strange. It's based on both n and val. More or less, this algorithm is O(n*val).

(a) Revised algorithm

Make the algorithm more efficient by using a divide-and-conquer strategy. You need not write working Java code, although you will receive a modicum of extra credit for writing working code.


/**
 * A divide-and-conquer square root algorithm.  Written as part of 
 * the answer key for HW3 in CSC152.2000S.  Note that the main method 
 * was copied nearly verbatim from SquareRoot.java, which was given 
 * in HW3.  Thanks to Paul Bailey for catching an error in the 
 * preconditions.
 *
 * @author Samuel A. Rebelsky
 * @version 1.0 of March 2000
 */
public class NewRoot
{
  // +------+----------------------------------------------------
  // | Main |
  // +------+

  public static void main(String[] args) {
    double val = 0;
    long n = 0;
    double root;
    SimpleOutput out = new SimpleOutput();   
    // Sanity check.
    if (args.length != 2) {
      out.println("Usage: java NewRoot value n");
      System.exit(1);
    }
    // Get the value.
    try { val = (new Double(args[0])).doubleValue(); }
    catch (NumberFormatException e) {
      out.println("The steps must be a number.");
      System.exit(2);
    } // catch
    // Get the n.
    try { n = (new Long(args[1])).longValue(); }
    catch (NumberFormatException e) {
      out.println("N must be an integer.");
      System.exit(3);
    } // catch
    // Verify that the preconditions are met.
    if (val < 1) {
      out.println("The value must be at least zero.");
      System.exit(4);
    } // if (val < 0)
    if (n <= 0) {
      out.println("N must be positive.");
    } // if (n <= 0)
    // Compute the square root
    root = sqrt(val,n);
    out.println("The square root of " + val + " is " + root);
    out.println("Java says that it is " + Math.sqrt(val));
  } // main(String[])


  // +---------+-------------------------------------------------
  // | Helpers |
  // +---------+

  /**
   * Compute the square root of val to accuracy 1/n.
   * Pre: (1) val >= 1
   *      (2) n >= 1
   *      (3) The square root of val can be represented.
   * Post: (1) Returns a value v such that |v-sqrt(val)| &lt; 1/n.
   */
  public static double sqrt(double val, long n) {
    // Compute 1/nth of the range of possible values
    double accuracy = 1.0/((double) n);
    // Set up a lower bound and an upper bound for the root
    double lowerBound = 0;
    double upperBound = n;
    // Determine a midpoint
    double guess = (lowerBound + upperBound)/2;
    // Keep going until things are accurate enough.
    while (upperBound-lowerBound > accuracy) {
      // If the guess is correct, then we're done.
      if (guess*guess == val)
        return guess;
      // If the guess is too small then
      else if (guess*guess < val) {
        // Make it the lower bound of possible values
        lowerBound = guess;
      }
      // Otherwise, the guess is too large
      else {
        // So make it the upper bound of possible values.
        upperBound = guess;
      }
      // Move on to the next guess
      guess = (lowerBound + upperBound) / 2;
    } // while
    // That's it, we're done
    return guess;
  } // sqrt(double, long)
} // class NewRoot


(b) Running time

Determine the running time of the revised algorithm.

We divide the range from 0 to val in half enough times that it becomes less than 1/n. This is the same as dividing the range from 0 to n*val in half enough times that it becomes 1. As you know, ``the number of times to divide x in half in order to get 1'' is log2x, so this is O(log2(n*val)).

3. Stamp Purchasing with Dynamic Programming

[This problem is based on a similar problem discussed in Duane Bailey's Java Structures.]

Suppose we have a package to send. We know how much it will cost to send the package. We'd like to minimize the number of stamps to purchase in order to send the package without paying extra. For example, if the package costs $0.40 to mail and the post office sells $0.50, $0.33, $0.20, $0.05, and $0.01 stamps, we'd purchase two $0.20 stamps (even though one $0.50 stamp would require fewer stamps).

Here's an approximate algorithm to determine the number of stamps we need. We assume that it is always possible to come up with a combination of stamps, no matter what value. (For example, we disallow the case when we want $0.07 and only $0.02 stamps are available.)

Our strategy is to see which stamp is best to buy now, and then go on (more or less). How do we know which is best to buy know? We recursively check how many steps it would cost if we bought each stamp and take the minimum of those numbers.

/**
 * Compute the minimum number of stamps that exactly
 * totals val cents.  
 * Pre: (1) val >= 0
 *      (2) All stamps values are positive.
 *      (3) It is possible to combine stamps for any value.
 * Post: Returns N such that it is possible to buy N stamps
 *   for exactly val and it is not possible to buy M < N 
 *   stamps for exactly val.
 */
public static int minimumStamps(
  int val, 
  int[] stampValues)
{
  // Base case: You need no stamps for 0 cents.
  if (val == 0) {
    return 0;
  }
  // Recursive case: Minimize alternatives
  else {
    int i = 0; // An index into stampValues
    int guess; // Our best guess so far as to the minimum.
    int nextGuess;  // Another guess to try
    int stampToBuy; // The stamp to buy in this round.

    // Find the first stamp that's still worth buying.
    while (stampValues[i] > val)
      i++;

    // We might buy that stamp.
    stampToBuy = stampValues[i];
    int guess = 1 + minimumStamps(val-stampValues[i], stampValues);

    // But we might also buy other stamps.
    for (i = i+1; i < stampValues.length; i++) {
      if (stampValues[i] <= val) {
        nextGuess = 
          1 + minimumStamps(val-stampValues[i], stampValues);
        if (nextGuess < guess) {
          stampToBuy = stampValues[i];
          guess = nextGuess;
        }
      } // if the stamp is worth buying.
    } // for
    
    // That's it, we're done
    return guess;
  } // recursive case
} // minimumStamps(int, int[])

(a) Informal analysis

Augment Stamps.java so that it counts the number of calls (recursive and otherwise) to minimumStamps. How many calls are made when you compute the number of stamps for 1, 5, 8, 10, 15, 20, 35, 40, 50, and 53 cents?

Changes (see NewerSampes.java):

Here are the results.

n stamps calls
1 1 1
2 2 3
5 1 7
8 4 19
10 2 33
15 3 139
20 1 571
35 3 39835
40 2 163,980
50 1 2,778,930
53 2 6,495,380

(b) Improving the Algorithm

As you may have noticed, this algorithm gets very slow as val gets large. Using the technique of caching smaller results in a table (a technique we call dynamic programming) that we used with such success in the box-packing and Fibonacci algorithms, rewrite this algorithm. You need not write working code, but it would be nice if you did.


import SimpleOutput;

/**
 * A very simple computation of the minimum number of stamps 
 * needed to total a particular price.  Based on Stamps.java,
 * given in HW3 of CSC152 2000S.  This extended version reports
 * on the number of function calls and uses a table to improve
 * the running time.
 *
 * @author Samuel A. Rebelsky
 * @version 1.0 of March 2000
 */
public class NewerStamps 
{
  // +--------+--------------------------------------------------
  // | Fields |
  // +--------+

  /** The number of recursive calls executed. */
  protected int numCalls;
  
  /** 
   * An array holding all the stamp values knows.  Has -1 in
   * each cell as a default.
   */
  protected int[] minStamps;
  
  // +---------+-------------------------------------------------
  // | Helpers |
  // +---------+

  /**
   * Compute the minimum number of stamps that exactly
   * totals val cents.  Puts the values of the stamps in
   * stampsToBuy, starting at index stampNum.
   * Pre: (1) val >= 0
   *      (2) All stamps values are positive.
   *      (3) It is possible to combine stamps for any value.
   *      (4) It requires no more than MAXSTAMPS stamps.
   * Post: Returns N such that it is possible to buy N stamps
   *   for exactly val and it is not possible to buy M < N 
   *   stamps for exactly val.
   */
  public int minimumStamps(int val, int[] stampValues)
  {
    // Increment the number of calls, since we're keeping track
    ++numCalls;
    // Preparation: Make sure that minStamps is defined.  This should
    // only be done once overall.
    if ((this.minStamps == null) || (this.minStamps.length <= val)) {
      this.minStamps = new int[val+1];
      for (int i = 1; i <= val; i++)
        minStamps[i] = -1;
      minStamps[0] = 0;
    }
    // Special case: minStamps[val] is not yet set.
    if (minStamps[val] < 0) {
      int i = 0; // An index into stampValues
      int guess; // Our best guess so far as to the minimum.
      int nextGuess;  // Another guess to try
      int buyThisTime; // The stamp to buy in this round.
  
      // Find the first stamp that's still worth buying.
      while (stampValues[i] > val)
        i++;
  
      // We might buy that stamp.
      buyThisTime = stampValues[i];
      guess = 1 + minimumStamps(val-stampValues[i], stampValues);
  
      // But we might also buy other stamps.
      for (i = i+1; i < stampValues.length; i++) {
        if (stampValues[i] <= val) {
          nextGuess =
            1 + minimumStamps(val-stampValues[i], stampValues);
          if (nextGuess < guess) {
            buyThisTime = stampValues[i];
            guess = nextGuess;
          } // if it's a better guess
        } // if the stamp is worth buying.
      } // for
      
      // Okay, we have a final solution
      minStamps[val] = guess;
    } // Special case
    
    // Okay, the array is now filled in!
    return minStamps[val];
  } // minimumStamps(int, int[])

  // +------+----------------------------------------------------
  // | Main |
  // +------+

  public static void main(String[] args) {
    int val = 0;
    SimpleOutput out = new SimpleOutput();   
    NewerStamps helper = new NewerStamps();
    // Set up the array of stamp values.
    int[] stamps = { 1, 5, 20, 33, 50 };

    // Sanity check.
    if (args.length != 1) {
      out.println("Usage: java Stamps value");
      System.exit(1);
    }
    // Get the value.
    try { val = (new Integer(args[0])).intValue(); }
    catch (NumberFormatException e) {
      out.println("The value must be an integer.");
      System.exit(2);
    } // catch
    // Verify that the preconditions are met.
    if (val < 0) {
      out.println("The value must be positive.");
      System.exit(3);
    } // if (val < 0)
    // Compute away!
    helper.numCalls = 0;
    int numStamps = helper.minimumStamps(val, stamps);
    if (numStamps == 0)
      out.println("You need no stamps to make no cents.");
    else if (numStamps == 1)
      out.println("You need one stamp to make " + val + " cents.");
    else
      out.println("You need " + numStamps + " stamps to make " 
                  + val + " cents.");
    out.println("That used " + helper.numCalls + " recursive calls.");
  } // main(String[])

} // class NewerStamps


Here are the revised results.

n stamps calls
1 1 2
2 2 3
5 5 7
8 4 13
10 2 17
15 3 27
20 1 38
35 3 86
40 2 106
50 1 147
53 2 162

For those of you interested in an iterative version, rather than a recursive version, here's one.


import SimpleOutput;

/**
 * A very simple iterative computation of the minimum
 * number of stamps needed to total a particular price.  
 * Based on a recursive version given as part of HW3 of
 * CSC152 2000S and rewritten for the answer key.
 *
 * @author Samuel A. Rebelsky
 * @version 1.0 of March 2000
 */
public class IterativeStamps 
{
  // +---------+-------------------------------------------------
  // | Helpers |
  // +---------+

  /**
   * Compute the minimum number of stamps that exactly
   * totals val cents.  Puts the values of the stamps in
   * stampsToBuy, starting at index stampNum.
   * Pre: (1) val >= 0
   *      (2) All stamps values are positive.
   *      (3) It is possible to combine stamps for any value.
   *      (4) It requires no more than MAXSTAMPS stamps.
   * Post: Returns N such that it is possible to buy N stamps
   *   for exactly val and it is not possible to buy M < N 
   *   stamps for exactly val.
   */
  public static int minimumStamps(int val, int[] stampValues)
  {
    // A friendly helper array.
    int[] minStamps = new int[val+1];
    minStamps[0] = 0;
    // Step through the possible values, setting the result as we go.
    for (int partialValue = 1; partialValue <= val; partialValue++) {
      int i = 0; // An index into stampValues
      int guess; // Our best guess so far as to the minimum.
      int nextGuess;  // Another guess to try
      int buyThisTime; // The stamp to buy in this round.
  
      // Find the first stamp that's still worth buying.
      while (stampValues[i] > partialValue)
        i++;
  
      // We might buy that stamp.
      buyThisTime = stampValues[i];
      guess = 1 + minStamps[partialValue-stampValues[i]];
  
      // But we might also buy other stamps.
      for (i = i+1; i < stampValues.length; i++) {
        if (stampValues[i] <= partialValue) {
          nextGuess =
            1 + minStamps[partialValue-stampValues[i]];
          if (nextGuess < guess) {
            buyThisTime = stampValues[i];
            guess = nextGuess;
          } // if it's a better guess
        } // if the stamp is worth buying.
      } // for
      
      // Okay, we have a final solution
      minStamps[partialValue] = guess;
    } // for each posible value
    
    // Okay, the array is now filled in!
    return minStamps[val];
  } // minimumStamps(int, int[])

  // +------+----------------------------------------------------
  // | Main |
  // +------+

  public static void main(String[] args) {
    int val = 0;
    SimpleOutput out = new SimpleOutput();   
    // Set up the array of stamp values.
    int[] stamps = { 1, 5, 20, 33, 50 };

    // Sanity check.
    if (args.length != 1) {
      out.println("Usage: java Stamps value");
      System.exit(1);
    }
    // Get the value.
    try { val = (new Integer(args[0])).intValue(); }
    catch (NumberFormatException e) {
      out.println("The value must be an integer.");
      System.exit(2);
    } // catch
    // Verify that the preconditions are met.
    if (val < 0) {
      out.println("The value must be positive.");
      System.exit(3);
    } // if (val < 0)
    // Compute away!
    int numStamps = minimumStamps(val, stamps);
    if (numStamps == 0)
      out.println("You need no stamps to make no cents.");
    else if (numStamps == 1)
      out.println("You need one stamp to make " + val + " cents.");
    else
      out.println("You need " + numStamps + " stamps to make " 
                  + val + " cents.");
  } // main(String[])

} // class IterativeStamps


History

Monday, 6 March 2000

Tuesday, 7 March 2000


Disclaimer Often, these pages were created "on the fly" with little, if any, proofreading. Any or all of the information on the pages may be incorrect. Please contact me if you notice errors.

This page may be found at http://www.math.grin.edu/~rebelsky/Courses/CS152/2000S/Assignments/notes.03.html

Source text last modified Tue Mar 7 17:07:57 2000.

This page generated on Wed Mar 8 08:23:08 2000 by Siteweaver. Validate this page's HTML.

Contact our webmaster at rebelsky@grinnell.edu