# Notes on Assignment 3: Algorithm Analysis

## Preliminaries

Do not read these notes until after you've turned in the assignment.

As you may have noted, I've released this answer key before grading your homeworks. That's because I think you'll want to refer to it as you work on the exam. I'll make some general comments on your assignments after I've graded them.

As many of you have noted, this is a lot like a long math assignment. Just as in math assignments you don't get all of the problems graded, you may find that I will not grade all of the problems on this assignment.

Here are the code files:

## 1. Big-O Computation

Bound the running time using Big-O notation of each of the following algorithms. Try to make the bound as close as possible. Note that you may have to come up with an appropriate metric for the size of each problem.

### (a) Smallest Value

```To find the smallest value in a collection ...
Set smallestSoFar to one value in the collection     O(1) steps
While unchecked values remain in the collection      O(n) repetitions
Pick one                                             O(1) steps
If that value is smaller than guess then             O(1) steps
Set smallestSoFar to that value                    O(1) steps
Return smallestSoFar                                 O(1) steps
```

So, the overall running time is O(2 + 3*n) which is O(n).

### (b) Smallest Value Using Divide and Conquer

We analyze this using recurrence relations. We'll let f(n) represent the running time on input of size n. Note that I've rewritten the algorithm slightly to clarify things.

```To find the smallest value in a collection ...
If the collection is empty then
Report an error
Otherwise, if the collection has only one value then    f(1) =
Return that value                                       1
Otherwise                                               f(n) =
Split the collection into two parts                     1 +
Find the smallest value in the first part               f(n/2) +
Find the smallest value in the second part              f(n/2) +
Take the minimum of those two values                    1
```

First, we expand the recursive version to see if we see a pattern.

```f(n) = 2 + 2*f(n/2)            // From above
= 2 + 2*(2 + 2*f(n/4))    // Expanded f(n/2)
= 2 + 4 + 4*f(n/4)        // Distributed 2*
= 6 + 4*f(n/4)            // Added 2+4
= 6 + 4*(2 + 2*f(n/8))    // Expanded f(n/4)
= 6 + 8 + 8*f(n/8)        // Distributed 2*
= 14 + 8*f(n/8)           // Added 6+8
= 14 + 8*(2 + 2*f(n/16))  // Expanded f(n/8)
= 14 + 16 + 16*f(n/16)    // Distributed 2*
= 30 + 16*f(n/16)
```

Okay, I see a pattern. I note that

```f(n) = 2*(2k - 1) + 2k*f(n/2k);
```

Now, I want the f term on the right to be f(1). This happens when

```n/2k = 1
n = 2k    // Multiply both sides by denominator
k = log2n // Definition of log
```

Hence

```f(n) = 2*(2log2n - 1) + 2log2n*f(1)
= 2*(n - 1) + n*f(1)
= 2n - 2 + n
= 3n -2
```

Therefore, f(n) is in O(n).

Note that we could have done a simpler, but less formal analysis by noting that we look at each value at most twice (once when we return it in the singleton and once when we take the minimum of two values). Hence, this is O(n).

Anne and Amber also noted that we don't really need the log in this program. Once we decide that 2k is equal to n, we can just substitute everywhere. However, the technique of determining the value of k is useful for other problems, so I left the analysis as I originally did it.

### (c) Least Difference

In the following, we'll let n be the number of values in the collection and p be the number of pairs.

```To find the smallest difference between any two
different numbers in a collection ...
List all pairs of different numbers in the collection  // O(p)
Set estimate to                                        // O(1)
the difference between the values in the first pair
For each remaining pair                                // O(p) repetitions
Compute the difference between the two values          // O(1)
If that differences is less than estimate then         // O(1)
Set estimate to that difference                      // O(1)
Return estimate                                        // O(1)
```

Hence, the result is O(p + 1 + 3*p + 1) or simply O(p).

Now, what is `p`? If there are `n` values in the collection, then there are O(n2) pairs. Hence this is an O(n2) algorithm.

### (d) Least Difference, Revisited

```To find the smallest difference between any two
different numbers in a collection ...
Set estimate to "infinity"               // O(1)
For each value, v, in the collection     // O(n) repetitions
For each value, u, not equal to v         // O(n) repetitions
If |u-v| < estimate then                   // O(1) steps
Set estimate to |u-v|                    // O(1) steps
Return estimate                          // O(1)
```

Hence, the running time is O(1 + n*n*2 + 1) which is O(n2).

## 2. Computing Square Roots with Divide and Conquer

Here's an iterative algorithm that finds the square root of a value to a specified accuracy.

```/**
* Compute the square root of val to accuracy 1/n.
* Pre: (1) val >= 0
*      (2) n >= 1
*      (3) The square root of val can be represented.
* Post: (1) Returns a value v such that |v-sqrt(val)| < 1/n.
*/
public static double sqrt(double val, long n) {
// Compute 1/nth of the range of possible values
double accuracy = 1.0/((double) n);
double guess = 0;
// Determine the difference between the square of the guess
// and the actual square root.
double diffsquared = val;

// Step through all the possible values between 0 and val.
for (double nextguess = 0.0;
nextguess <= val;
nextguess = nextguess + accuracy) {
// Compute the difference between the square of the guess
// and the actual value.
double nextdiff = Math.abs(nextguess*nextguess-val);
// If it's better, use it.
if (nextdiff < diffsquared) {
guess = nextguess;
diffsquared = nextdiff;
}
} // for
return guess;
} // sqrt(double,double)
```

As you might be able to tell, what this algorithm basically does is divide the interval between 0 and val into potential guesses that occur every 1/n units. It then tries each of these guesses and uses the best. Why 0 and val? Because we know that the square root of val is at least 0 and no more than val.

The running time of this algorithm is somewhat strange. It's based on both n and val. More or less, this algorithm is O(n*val).

### (a) Revised algorithm

Make the algorithm more efficient by using a divide-and-conquer strategy. You need not write working Java code, although you will receive a modicum of extra credit for writing working code.

```
/**
* A divide-and-conquer square root algorithm.  Written as part of
* the answer key for HW3 in CSC152.2000S.  Note that the main method
* was copied nearly verbatim from SquareRoot.java, which was given
* in HW3.  Thanks to Paul Bailey for catching an error in the
* preconditions.
*
* @author Samuel A. Rebelsky
* @version 1.0 of March 2000
*/
public class NewRoot
{
// +------+----------------------------------------------------
// | Main |
// +------+

public static void main(String[] args) {
double val = 0;
long n = 0;
double root;
SimpleOutput out = new SimpleOutput();
// Sanity check.
if (args.length != 2) {
out.println("Usage: java NewRoot value n");
System.exit(1);
}
// Get the value.
try { val = (new Double(args[0])).doubleValue(); }
catch (NumberFormatException e) {
out.println("The steps must be a number.");
System.exit(2);
} // catch
// Get the n.
try { n = (new Long(args[1])).longValue(); }
catch (NumberFormatException e) {
out.println("N must be an integer.");
System.exit(3);
} // catch
// Verify that the preconditions are met.
if (val < 1) {
out.println("The value must be at least zero.");
System.exit(4);
} // if (val < 0)
if (n <= 0) {
out.println("N must be positive.");
} // if (n <= 0)
// Compute the square root
root = sqrt(val,n);
out.println("The square root of " + val + " is " + root);
out.println("Java says that it is " + Math.sqrt(val));
} // main(String[])

// +---------+-------------------------------------------------
// | Helpers |
// +---------+

/**
* Compute the square root of val to accuracy 1/n.
* Pre: (1) val >= 1
*      (2) n >= 1
*      (3) The square root of val can be represented.
* Post: (1) Returns a value v such that |v-sqrt(val)| &lt; 1/n.
*/
public static double sqrt(double val, long n) {
// Compute 1/nth of the range of possible values
double accuracy = 1.0/((double) n);
// Set up a lower bound and an upper bound for the root
double lowerBound = 0;
double upperBound = n;
// Determine a midpoint
double guess = (lowerBound + upperBound)/2;
// Keep going until things are accurate enough.
while (upperBound-lowerBound > accuracy) {
// If the guess is correct, then we're done.
if (guess*guess == val)
return guess;
// If the guess is too small then
else if (guess*guess < val) {
// Make it the lower bound of possible values
lowerBound = guess;
}
// Otherwise, the guess is too large
else {
// So make it the upper bound of possible values.
upperBound = guess;
}
// Move on to the next guess
guess = (lowerBound + upperBound) / 2;
} // while
// That's it, we're done
return guess;
} // sqrt(double, long)
} // class NewRoot

```

### (b) Running time

Determine the running time of the revised algorithm.

We divide the range from 0 to val in half enough times that it becomes less than 1/n. This is the same as dividing the range from 0 to n*val in half enough times that it becomes 1. As you know, ``the number of times to divide x in half in order to get 1'' is log2x, so this is O(log2(n*val)).

## 3. Stamp Purchasing with Dynamic Programming

[This problem is based on a similar problem discussed in Duane Bailey's Java Structures.]

Suppose we have a package to send. We know how much it will cost to send the package. We'd like to minimize the number of stamps to purchase in order to send the package without paying extra. For example, if the package costs \$0.40 to mail and the post office sells \$0.50, \$0.33, \$0.20, \$0.05, and \$0.01 stamps, we'd purchase two \$0.20 stamps (even though one \$0.50 stamp would require fewer stamps).

Here's an approximate algorithm to determine the number of stamps we need. We assume that it is always possible to come up with a combination of stamps, no matter what value. (For example, we disallow the case when we want \$0.07 and only \$0.02 stamps are available.)

Our strategy is to see which stamp is best to buy now, and then go on (more or less). How do we know which is best to buy know? We recursively check how many steps it would cost if we bought each stamp and take the minimum of those numbers.

```/**
* Compute the minimum number of stamps that exactly
* totals val cents.
* Pre: (1) val >= 0
*      (2) All stamps values are positive.
*      (3) It is possible to combine stamps for any value.
* Post: Returns N such that it is possible to buy N stamps
*   for exactly val and it is not possible to buy M < N
*   stamps for exactly val.
*/
public static int minimumStamps(
int val,
int[] stampValues)
{
// Base case: You need no stamps for 0 cents.
if (val == 0) {
return 0;
}
// Recursive case: Minimize alternatives
else {
int i = 0; // An index into stampValues
int guess; // Our best guess so far as to the minimum.
int nextGuess;  // Another guess to try

// Find the first stamp that's still worth buying.
while (stampValues[i] > val)
i++;

// We might buy that stamp.
int guess = 1 + minimumStamps(val-stampValues[i], stampValues);

// But we might also buy other stamps.
for (i = i+1; i < stampValues.length; i++) {
if (stampValues[i] <= val) {
nextGuess =
1 + minimumStamps(val-stampValues[i], stampValues);
if (nextGuess < guess) {
guess = nextGuess;
}
} // if the stamp is worth buying.
} // for

// That's it, we're done
return guess;
} // recursive case
} // minimumStamps(int, int[])
```

### (a) Informal analysis

Augment `Stamps.java` so that it counts the number of calls (recursive and otherwise) to `minimumStamps`. How many calls are made when you compute the number of stamps for 1, 5, 8, 10, 15, 20, 35, 40, 50, and 53 cents?

Changes (see `NewerSampes.java`):

• Added a `numCalls` field to the class
```  // +--------+--------------------------------------------------
// | Fields |
// +--------+

/** The number of recursive calls executed. */
protected int numCalls;
```
• At the start of `minimumStamps`, added a line to increment `numCalls`
```  public int minimumStamps(int val, int[] stampValues)
{
// Increment the number of calls, since we're keeping track
++numCalls;
```
• In `main`, before the first call to `minimumStamps`, set `helper.numCalls` to 0.
```    // Compute away!
helper.numCalls = 0;
int numStamps = helper.minimumStamps(val, stamps);
```
• At the end of `main`, print out `helper.numCalls`.
```    out.println("That used " + helper.numCalls + " recursive calls.");
```

Here are the results.

n stamps calls
1 1 1
2 2 3
5 1 7
8 4 19
10 2 33
15 3 139
20 1 571
35 3 39835
40 2 163,980
50 1 2,778,930
53 2 6,495,380

### (b) Improving the Algorithm

As you may have noticed, this algorithm gets very slow as val gets large. Using the technique of caching smaller results in a table (a technique we call dynamic programming) that we used with such success in the box-packing and Fibonacci algorithms, rewrite this algorithm. You need not write working code, but it would be nice if you did.

```
import SimpleOutput;

/**
* A very simple computation of the minimum number of stamps
* needed to total a particular price.  Based on Stamps.java,
* given in HW3 of CSC152 2000S.  This extended version reports
* on the number of function calls and uses a table to improve
* the running time.
*
* @author Samuel A. Rebelsky
* @version 1.0 of March 2000
*/
{
// +--------+--------------------------------------------------
// | Fields |
// +--------+

/** The number of recursive calls executed. */
protected int numCalls;

/**
* An array holding all the stamp values knows.  Has -1 in
* each cell as a default.
*/
protected int[] minStamps;

// +---------+-------------------------------------------------
// | Helpers |
// +---------+

/**
* Compute the minimum number of stamps that exactly
* totals val cents.  Puts the values of the stamps in
* stampsToBuy, starting at index stampNum.
* Pre: (1) val >= 0
*      (2) All stamps values are positive.
*      (3) It is possible to combine stamps for any value.
*      (4) It requires no more than MAXSTAMPS stamps.
* Post: Returns N such that it is possible to buy N stamps
*   for exactly val and it is not possible to buy M < N
*   stamps for exactly val.
*/
public int minimumStamps(int val, int[] stampValues)
{
// Increment the number of calls, since we're keeping track
++numCalls;
// Preparation: Make sure that minStamps is defined.  This should
// only be done once overall.
if ((this.minStamps == null) || (this.minStamps.length <= val)) {
this.minStamps = new int[val+1];
for (int i = 1; i <= val; i++)
minStamps[i] = -1;
minStamps[0] = 0;
}
// Special case: minStamps[val] is not yet set.
if (minStamps[val] < 0) {
int i = 0; // An index into stampValues
int guess; // Our best guess so far as to the minimum.
int nextGuess;  // Another guess to try

// Find the first stamp that's still worth buying.
while (stampValues[i] > val)
i++;

// We might buy that stamp.
guess = 1 + minimumStamps(val-stampValues[i], stampValues);

// But we might also buy other stamps.
for (i = i+1; i < stampValues.length; i++) {
if (stampValues[i] <= val) {
nextGuess =
1 + minimumStamps(val-stampValues[i], stampValues);
if (nextGuess < guess) {
guess = nextGuess;
} // if it's a better guess
} // if the stamp is worth buying.
} // for

// Okay, we have a final solution
minStamps[val] = guess;
} // Special case

// Okay, the array is now filled in!
return minStamps[val];
} // minimumStamps(int, int[])

// +------+----------------------------------------------------
// | Main |
// +------+

public static void main(String[] args) {
int val = 0;
SimpleOutput out = new SimpleOutput();
// Set up the array of stamp values.
int[] stamps = { 1, 5, 20, 33, 50 };

// Sanity check.
if (args.length != 1) {
out.println("Usage: java Stamps value");
System.exit(1);
}
// Get the value.
try { val = (new Integer(args[0])).intValue(); }
catch (NumberFormatException e) {
out.println("The value must be an integer.");
System.exit(2);
} // catch
// Verify that the preconditions are met.
if (val < 0) {
out.println("The value must be positive.");
System.exit(3);
} // if (val < 0)
// Compute away!
helper.numCalls = 0;
int numStamps = helper.minimumStamps(val, stamps);
if (numStamps == 0)
out.println("You need no stamps to make no cents.");
else if (numStamps == 1)
out.println("You need one stamp to make " + val + " cents.");
else
out.println("You need " + numStamps + " stamps to make "
+ val + " cents.");
out.println("That used " + helper.numCalls + " recursive calls.");
} // main(String[])

```

Here are the revised results.

n stamps calls
1 1 2
2 2 3
5 5 7
8 4 13
10 2 17
15 3 27
20 1 38
35 3 86
40 2 106
50 1 147
53 2 162

For those of you interested in an iterative version, rather than a recursive version, here's one.

```
import SimpleOutput;

/**
* A very simple iterative computation of the minimum
* number of stamps needed to total a particular price.
* Based on a recursive version given as part of HW3 of
* CSC152 2000S and rewritten for the answer key.
*
* @author Samuel A. Rebelsky
* @version 1.0 of March 2000
*/
public class IterativeStamps
{
// +---------+-------------------------------------------------
// | Helpers |
// +---------+

/**
* Compute the minimum number of stamps that exactly
* totals val cents.  Puts the values of the stamps in
* stampsToBuy, starting at index stampNum.
* Pre: (1) val >= 0
*      (2) All stamps values are positive.
*      (3) It is possible to combine stamps for any value.
*      (4) It requires no more than MAXSTAMPS stamps.
* Post: Returns N such that it is possible to buy N stamps
*   for exactly val and it is not possible to buy M < N
*   stamps for exactly val.
*/
public static int minimumStamps(int val, int[] stampValues)
{
// A friendly helper array.
int[] minStamps = new int[val+1];
minStamps[0] = 0;
// Step through the possible values, setting the result as we go.
for (int partialValue = 1; partialValue <= val; partialValue++) {
int i = 0; // An index into stampValues
int guess; // Our best guess so far as to the minimum.
int nextGuess;  // Another guess to try

// Find the first stamp that's still worth buying.
while (stampValues[i] > partialValue)
i++;

// We might buy that stamp.
guess = 1 + minStamps[partialValue-stampValues[i]];

// But we might also buy other stamps.
for (i = i+1; i < stampValues.length; i++) {
if (stampValues[i] <= partialValue) {
nextGuess =
1 + minStamps[partialValue-stampValues[i]];
if (nextGuess < guess) {
guess = nextGuess;
} // if it's a better guess
} // if the stamp is worth buying.
} // for

// Okay, we have a final solution
minStamps[partialValue] = guess;
} // for each posible value

// Okay, the array is now filled in!
return minStamps[val];
} // minimumStamps(int, int[])

// +------+----------------------------------------------------
// | Main |
// +------+

public static void main(String[] args) {
int val = 0;
SimpleOutput out = new SimpleOutput();
// Set up the array of stamp values.
int[] stamps = { 1, 5, 20, 33, 50 };

// Sanity check.
if (args.length != 1) {
out.println("Usage: java Stamps value");
System.exit(1);
}
// Get the value.
try { val = (new Integer(args[0])).intValue(); }
catch (NumberFormatException e) {
out.println("The value must be an integer.");
System.exit(2);
} // catch
// Verify that the preconditions are met.
if (val < 0) {
out.println("The value must be positive.");
System.exit(3);
} // if (val < 0)
// Compute away!
int numStamps = minimumStamps(val, stamps);
if (numStamps == 0)
out.println("You need no stamps to make no cents.");
else if (numStamps == 1)
out.println("You need one stamp to make " + val + " cents.");
else
out.println("You need " + numStamps + " stamps to make "
+ val + " cents.");
} // main(String[])

} // class IterativeStamps

```

## History

Monday, 6 March 2000

• Created first version.

Tuesday, 7 March 2000

• Updated and corrected. Placed online.

Disclaimer Often, these pages were created "on the fly" with little, if any, proofreading. Any or all of the information on the pages may be incorrect. Please contact me if you notice errors.