Held Wednesday, September 29, 1999
Overview
Today, we move from details of Java to consideration of the ways
in which one might compare and analyze algorithms. We will ground
our analysis in a few sample algorithms for finding the smallest
value in a collection.
Notes
- Some notes on the exam
- A few of you asked about the median and mean grades of the exam.
The median grade was 83. The mean grade was 85.
- A few of you asked about whether I curve grades. I don't.
- A few of you asked about my grading scale. 94 and up is an A.
90-93 is an A-. 87-89 is a B+. 84-86 is a B. And so on and
so forth.
- Rich Salter of Oberlin is talking in on Friday at noon in 2413 on
some interesting work in computers and education (and on hypertext
in general). Bring a lunch and enjoy.
- For tomorrow, start reading Chapter 5 on recursion and read lab A1
(surprisingly, also on Recursion).
- A few of you noted that no one is working on the Mailbox class.
That's an oversight on my part. If no one else wants to work on
the class, I'll do it.
- I encourage you to consider taking the annual Putnam Exam,
a fairly-difficult nationwide competition in problem-solving.
The exam will take place Saturday, December 4. Dr. Adelberg has to return the
registration forms to be received by October 15, so anybody interested in
taking the exam should inform him as soon as possible (office 2405, e-mail
adelbe@math.grin.edu).
Contents
Handouts
Summary
- Two simple ``smallest'' algorithms
- Basics of algorithm analysis
- Due:
- Section 5.5: Comparing Algorithms
- Assigned:
- Chapter 5 (should be 6): Recursion
- Lab A1: Recursion
- You may recall from the first day of class that the foci of CS152
are algorithms and data structures.
- Algorithms are sets of instructions for completing tasks.
- Data structures are mechanisms that organize information.
- We will begin with some consideration of algorithm analysis.
- To help, we'll consider two algorithms for finding the smallest
integer in a collection of values. (Nope, no Java.)
- Here's the first algorithm:
Pick a value in the collection. Make that our initial guess.
While there are values we haven't looked at
Pick a value we haven't looked at.
If that value is smaller than our guess, update our guess.
Once we've looked at all the values, return our guess.
- Here's the second algorithm:
If the collection has only one element, then
Return that element
Otherwise
Split the collection in half.
Find the smallest element in each half.
Return the minimum of those two.
- As you may have noted, there are often multiple algorithms one can
use to solve the same problem.
- In searching an ordered list, one can use linear search, binary
search, or ``look randomly'' (as well as many others). [Don't
worry if you don't know any of these.]
- In finding the minimum element of a list, you can step through
the list, keeping track of the current minimum. You could also
sort the list and grab the first element.
- You can come up with your own variants.
- How do we choose which algorithm is the best?
- The fastest/most efficient algorithm.
- The one that uses the fewest resources.
- The clearest.
- The shortest.
- The easiest to write.
- The most general.
- ...
- Frequently, we look at the ``speed''. That is, we consider how
long the algorithm takes to run.
- It is therefore important for us to be able to analyze the running
time of our algorithms.
Is there an exact number we can provide for the running
time of an algorithm? Surprisingly, no.
- Different inputs lead to different running times. For example,
if there are conditionals in the algorithm (as there are in both of
our smallest algorithms), different instructions will be executed depending
on the input.
- Not all operations take the same time. For example, addition is
typically quicker than multiplication, and integer addition is
typically quicker than floating point addition.
- The same operation make take different times on different machines.
- The same operation make appear to take different times on the same machine,
particularly if other things are happening on the same machine.
- Many things that affect running time happen behind the scenes and
cannot be easily predicted. For example, the computer might move
some frequently-used data to cache memory.
- Noting problems in providing a precise analysis of the running time of
programs, computer scientists developed a technique which
is often called asymptotic analysis. In asymptotic analysis of
algorithms, one describes the general behavior of algorithms in terms of
the size of input, but without delving into precise details.
- The analysis is ``asymptotic'' in that we look at the behavior
as the input gets larger.
- There are many issues to consider in analyzing the asymptotic behavior
of a program. One particularly useful metric is an upper bound
on the running time of an algorithm. We call this the ``Big-O'' of
an algorithm.
- Big-O is defined somewhat mathematically, as a relationship between
functions.
- f(n) is in O(g(n)) iff
- there exists a number n0
- there exists a number d > 0
- for all n > n0, abs(f(n)) <= abs(d*g(n))
- What does this say? It says that after a certain value, n0, f(n) is
bounded above by a constant (d) times g(n).
- The constant (d) helps accommodate the variation in the algorithm.
- We don't usually identify the d precisely.
- The n0 says ``for big enough n''.
- We can apply big-O to algorithms.
- n is the "size" of the input (e.g., the number of items in a list
or vector to be manipulated).
- f(n) is the running time of the algorithm.
- Some common Big-O bounds
- An algorithm that is O(1) takes constant time. That is, the
running time is independent of the input. Getting the size of an
array should be an O(1) algorithm.
- An algorithm that is O(n) takes time linear in the size of the
input. That is, we basically do constant work for each ``element'' of
the input. Finding the smallest element in a list is often an
O(n) algorithm.
- An algorithm that is O(log_2(n)) takes logarithmic time.
While the running time is dependent on the size of the input,
it is clear that not every element of the input is processed.
Many such algorithms involve the strategy of ``divide and conquer''.
- One of the nice things about asymptotic analysis is that it makes
constants ``unimportant'' because they can be ``hidden'' in the d.
- If f(n) is 100*n seconds and g(n) is 0.5*n seconds, then
f(n) is in O(g(n)) [let d be 200] and g(n) is in f(n).
- If f(n) is 100*n seconds and g(n) is n*n seconds, then
f(n) is O(g(n)) [let n0 be 100 and d be 1; let n0 be 1 and d be 100; ...].
- However, g(n) is not O(f(n)). Why not? Suppose there were an n0 and
a d. Consider what happens for n = 101d. d*f(n) = d*100*101*d =
d*d*100*101. However, g(n) = d*d*101*101, which is even larger.
If n0 is greater than 101d, we'll still have this problem [proof left
to reader].
- Since constants can be eliminated, we normally don't write them.
- That is, we say that the running time of an algorithm is
O(n) or O(n^{2}), ....
- We now have a theoretical grounding for asymptotic analysis. How
do we do it in practice?
- At this point in your career, it's often best to ``count'' the steps
in an algorithm and then add them up. After you've taken combinatorics,
you can use recurrence relations.
- Over the next few days, we'll look at a number of examples. Some
starting ones.
- Finding the smallest/largest element in an array of integers.
- Finding the average of all the elements in an array of integers.
- Putting the largest element in an array at the end of the array.
if we're only allowed to swap subsequent elements.
- Computing the nth Fibonacci number.
- ...
- Let us begin by considering the two algorithms we introduced earlier.
- What can we say about the first algorithm?
- What can we say about the second?
- Although the previous discussion may make it seem like precise
details of algorithms aren't important, some details are
particularly important.
- Consider our recursive ``find the smallest value'' algorithm.
Suppose we stored the values in a list.
- How do we split the list?
- How long should that take?
- Is that a constant?
Tuesday, 10 August 1999
- Created as a blank outline.
Monday, 27 September 1999
- Filled in the grading strategy for exam 1 and the note on Rich Salter.
Tuesday, 28 September 1999
- Filled in the details from
outline 15 of
CS152 99S.
- Reformatted and modified some text.
- Added introductory section.
- Added section about details.
- Added overview comment.
Wednesday, 29 September 1999
Back to Discussion of Exam 1 and Project.
On to Recursion.