Outline of Class 24: Introduction to Sorting

Held: Tuesday, March 3, 1998

Administrivia

• We won't go over Bailey's stamps problem, as it is very similar to the Fibonacci problem. Let me know if you have questions.
• So, who has an answer for yesterday's question: Can you come up with a more efficient way to compute the nth Fibonacci number?
• Are there any questions on assignment 4?
• If you done so already, please start to read the section of our text on sorting (chapter 5).

Dynamic Programming

• In many recursively defined functions, it turns out that you end up making the same function call (that is, a call to the same function with the same arguements) many times.

• Consider the typical solution to the problem of finding the nth Fibonacci number.
• The Fibonacci numbers are defined as:
  • 1 is the 0th Fibonacci number
  • 1 is the 1st Fibonacci number
  • The nth Fibonacci number is the n-1st Fibonacci number plus the n-2nd Fibonacci number.
• The sequence begins 1, 1, 2, 3, 5, 8, 13, 21, ...
• (The ratio of the n+1st to nth Fibonacci number approaches the golden ratio.)
• One might code this as

  /**
     Compute the nth Fibonacci number.
     pre: n >= 0
     pre: fib(n) <= maximum integer
     post: returns the nth fibonacci number
   */
  public static int fib(int n) {
    // Base case
    if ((n == 0) || (n == 1)) {
      return 1;
    } // base case
    // Recursive case
    else {
      return fib(n-1) + fib(n-2);
    } // recursive case
  } // fib(n)
  

• Unfortunately, this has a lot of repeated work. To compute fib(8) we have one call to fib(7) + one call to fib(6)
  • Which is two calls to fib(6) + one call to fib(5)
  • Which is three calls to fib(5) + two calls to fib(4)
  • Which is five calls to fib(4) + three calls to fib(3)
  • Which is eight calls to fib(3) + five calls to fib(2)
  • Which is thirteen calls to fib(2) + eight calls to fib(1)
  • Which is twenty-one calls to fib(1) + thirteen calls to fib(0)
  • Which is 34 calls to the base case
• This is clearly inefficient.
• But our analysis gives us a clue as to a better way to compute the answer: build an array of solutions, and work our way up.

  /**
     Compute the nth Fibonacci number.
     pre: n >= 0
     pre: fib(n) <= maximum integer
     post: returns the nth fibonacci number
   */
  public static int newfib(int n) {
    int[] solutions = new int[n+1];
    // Fill in the intial solutions
    solutions[0] = 1;
    solutions[1] = 1;
    // Fill in the remaining solutions
    for(int i = 2; i <=n; ++i) {
      solutions[i] = solutions[i-1] + solutions[i-2]
    } // for
    // That's it
    return solutions[n];
  } // newfib
  

• Of course, this isn't recursive. If we wanted to rewrite it recursively, we might pass the array of solutions to our recursive calls.

  /**
     Compute the nth Fibonacci number.
     pre: n >= 0
     pre: fib(n) <= maximum integer
     post: returns the nth fibonacci number
   */
  public static int newerfib(int n) {
    int[] solutions = new int[n+1];
    // We'll use 0 to indicate "no solution yet"
    for(int i = 0; i <= n; ++i) {
      solutions[i] = 0;
    }
    // Compute the fibonacci using this helper array
    return newerfib(n,solutions);
  } // newerfib(int)
   
  /**
     Compute the nth Fibonacci number.
     pre: n >= 0
     pre: fib(n) <= maximum integer
     pre: solutions.length >= n+1
     post: returns the nth fibonacci number
   */
  public static int newerfib(int n, int[] solutions) {
    // Deal with precomputed solutions.  Note that return
    // immediately exits the routine, so I don't need an 
    // else clause.
    if (solutions[n] != 0) {
      return solutions[n];
    }
    // Base case
    if ((n == 0) || (n == 1)) {
      solutions[n] = 1;
    } // base case
    // Recursive case
    else {
      solutions[n] = newerfib(n-1,solutions) + newerfib(n-2,solutions);
    } // recursive case
    return solutions[n];
  } // newerfib(int,int[])
  

• What's the new running time? It's O(n). However, this may require more space, as we need to allocate an array of size theta(n).
• Yes, there are other ways to optimize the Fibonacci calculuation. For example, it turns out that you can simple keep track of the current two values.

• In the technique of dynamic programming, you use arrays of cached solutions, as above. It turns out that this is a good solution technique for many classes of problems, particularly problems that require us to consider a number of variations.

Sorting

• Typically, computer scientists look at collections of problems and attempt to generalize subproblems. They then look for optimal solutions to those subproblems.
• One problem that seems to crop up a lot is that of sorting. Given a list, array, or vector of comparable elements, put the elements in order.
  • in order means that each element is no bigger than the next element. (You can also sort in decreasing order, in which case each element is no smaller than the next element.)
  • you usually need to ensure that all elements in the original list are in the sorted list.
• In evaluating sorting methods, we should concern ourselves with both the running time and the amount of extra storage (beyond the original vector) that is required.
• In place sorting is a special subclass of sorting algorithms in which the original object is modified, and little, if any, extra storage is used.
• Most often, sorting is accomplished by repeatedly swapping elements of the vector. However, this is not the only way in which sorting can be done.

Selection sort

• Selection sort is among the simpler and more natural methods for sorting.
• In this sorting algorithm, you break the vector up into two subvectors, a sorted part and an unsorted part. You repeatedly find the largest of the unsorted elements in the Vector, and put that at the beginning of the sorted part. This continues until there are no unsorted elements.
• Here's my version of selection sort. It is a method of a Vector-like object, so we don't explicity refer to the vector.

  /**
   * Sort all the elements in the subvector between lb and ub.
   * pre: 0 <= lb <= ub < size()
   * pre: the elements in the vector are comparable using
   *      a lessEqual method
   * post: elementAt(lb) <= elementAt(lb+1) <= ... elementAt(ub)
   * post: no element is added to or removed from the vector
   */
  public void selectionSort(int lb, int ub) 
  {
     // Variables
     int index_of_largest;	// ... element in subrange
     // The preconditions will be checked in the following
     // function call, so we don't check them here
  
     // Base case: one element, so it's sorted
     if (lb == ub) {
       return;
     }
     // Find the index of the largest element in the subrange
     index_of_largest = indexOfLargest(lb,ub);
     // Swap that element and the last element
     swap(index_of_largest, ub);
     // Sort the rest of the subvector (if there is any)
     // Note that we don't have to compare ub-1 to lb, since
     //   the preconditions and the base case take care of it.
     selection_sort(lb,ub-1);
  } // selectionSort
  
  /**
   * Find the index of the largest element in a subvector
   * pre: 0 <= lb <= ub < size()
   * pre: the elements in the vector are comparable using
   *      a lessEqual method
   * post: returns I s.t. for all i, lb <= i <= ub, 
   *       elementAt(I) >= elementAt(i)
   */
  public int indexOfLargest(int lb, int ub) {
     // Variables
     int guess;	// Current guess as to index of largest
     // Check the preconditions
     Assert.pre(lb <= ub, "nonempty subrange");
     Assert.pre(0 <= lb, "reasonable starting point");
     Assert.pre(ub < size(), "reasonable ending point");
     // Make initial guesses
     guess = lb;
     // Repeatedly improve our guesses until we've looked at
     // all the elements
     for(int i = lb+1; i <= lb; ++i) {
       if (elementAt(guess).lessEqual(elementAt(i))) {
          guess = i;
       } // if
     } // for
     // That's it
     return guess;
  } // index_of_largest
  

• What's the running time of this algorithm? To sort a vector of n elements, we have to find the largest element in that vector in O(n) steps, and then recurse on the rest. The first recursive call takes O(n-1) steps plus the recursion. And so on and so forth. This makes it an O(n^2) algorithm.
• What's the extra memory required by this algorithm (ignoring the extra memory for recursive calls)? It's more or less O(1), since we only allocate a few extra variables and no extra vectors.
• How much extra memory is required for recursive method calls? This is a tail-recursive algorithm, so there shouldn't be any.