Class 53: Shortest Path, Revisited; Graphs, Concluded

Held Monday, May 10

Summary

• Dijkstra's algorithm, in depth
• A few more algorithms
• Minimum spanning tree

Contents

Handouts

• About end-of-course forms
• End-of-course survey
• Departmental course evaluation form

Notes

• A reminder: attendance is required for the rest of the semester.
• Are there questions on the exam?
  • It is still due on Thursday.
• Don't forget: at noon today we'll be doing our demonstrations. I've prepared a rough schedule.
  • We'll use the last twenty minutes or so of today's class to get ready.
  • Clif will lead that discusson; I'll go get pizza.
• I've included notes on today's class. However, I'd prefer that you not refer to them until after class, so that we can continue our quality discussions.
• Reminder: On Friday, May 14, 1999, each group should turn in a four page report on your project. It should summarize the design decisions, give enough information for other people to use your project, and also give enough information for other people to modify your project.
• Each student is also responsible for a one page report reflecting on the process by which you built the project (what was good, what was bad, who deserves the credit or blame, etc.).
• I realize that most of your projects are not yet working perfectly. I'm willing to hear suggestions on what the due date should be.
• I've updated the grading policies to accomodate the project.
  • Exams: 30% (3 graded; final is a makeup)
  • Homework: 40% (3 graded)
  • Project: 20%
  • Best-of-three: 10% (whichever category you did best on)

Dijkstra's Shortest Path Algorithm

• A computer scientist named Dijkstra proposed an interesting strategy for finding the shortest path from A to B.
• He suggested that we can find the shortest path from A to B by finding the shortest path from A to all nodes in the graph. (Talk about overkill.)
• His algorithm is generally used with the ``sum of weights'' metric.
  • It may also work with selected other metrics.
• His algorithm assumes that edge weights are nonnegative.
• We'll divide the graph into two parts:
  • SP. The nodes whose shortest path we know. For these nodes, we'll store the distance to that node and the path to that node.
  • Est. The nodes whose shortest path we don't know. For these nodes, we'll store the shortest known distance to that node (which may not be the shortest) and the corresponding path.
• Initially, Est contains all the nodes. The distance from A to itself is 0, and the distance from A to every other node is some distance greater than the largest distance in the graph.
• At each step, we'll move one node from Est to SP. After doing so, we'll update the estimated costs of all of its neighbors.
• Which node to we move? Node s, the one with the smallest estimated distance.
• Why is this safe? Because the only way to reduce that distance would be to pick another node, t from Est and have a path from that node to s such that A to t to s has smaller distance than our current estimate of the distance form A to s. But that's not possible, since the cost function is non-decreasing, and we already know that the cost from A to t is at least as big as the cost from A to s.
• Note that instead of storing the path, we can simply store the previous node in the path.
• Dijkstra's algorithm is another example of a greedy algorithm: at each step in the algorithm, we pick a ``best'' node

An Example

• Here's a sample graph to consider. I've used an undirected graph because the algorithm works equally well for undirected graphs and they're easier to draw.

      G
      |
     3|
      | 1   4   1   9
      A---B---C---D---E
      |   |   |       |
     9|  2|  0|       |
      |   |   |       |
      +---F---+       |
          |      6    |
          +-----------+
  

• What is the shortest path from A to E? It may be hard to see at first (not too hard, but nontrivial).
• Let's see what the algorithm tells us.
• Initially,
  • SP = { }
  • Est = { A:(0,empty) B:(100,?) C:(100,?) D:(100,?) E:(100,?) F:(100,?) G:(100,?) }
• The smallest distance in that graph is the distance to A, so we
  • Move A to SP
  • Update the distances to all of its neighbors (B, F, and G which now have actual paths and distances)
• We now have the following information
  • SP = { A:(0,empty) }
  • Est = { B:(1,A) G:(3,A) F:(9,A) C:(100,?) D:(100,?) E:(100,?) }
• We move B to SP and update its neighbors. We now know a path to C and a better path to F (A->B->F has cost 1 + 2 = 3).
  • SP = { A:(0,empty) B:(1,A) }
  • Est = { F:(3,B) G:(3,A) C:(5,B) D:(100,?) E:(100,?) }
• We could then move F or G to SP. Let's say that we move F. We now know a shorter path to C (A->B->F->C has cost 3, A->B->C had cost 5) and a path to E.
  • SP = { A:(0,empty) B:(1,A) F:(3,B) }
  • Est = { G:(3,A) C:(3,F) E:(9,F) D:(100,?) }
• We can then move G or C to SP. Let's say that we move G. G has no neighbors, so there are no other changes.
  • SP = { A:(0,empty) B:(1,A) F:(3,B) G:(3,A) }
  • Est = { C:(3,F) E:(9,F) D:(100,?) }
• Next we move C. We can then update the distance to D.
  • SP = { A:(0,empty) B:(1,A) F:(3,B) G:(3,A) C:(3,F) }
  • Est = { D:(4,C) E:(9,F) }
• We then move D. Note that even though there is an edge from D to E, it doesn't give us any better path, so we don't change the entry for E.
  • SP = { A:(0,empty) B:(1,A) F:(3,B) G:(3,A) C:(3,F) D:(4,C) }
  • Est = { E:(9,F) }
• We move E and we're done
  • SP = { A:(0,empty) B:(1,A) F:(3,B) G:(3,A) C:(3,F) D:(4,C) E:(9,F) }
  • Est = { }
• What's the shortest path from A to E? It has cost 9 and is the reverse of (E-F-B-A). How did I get that path? I read it off from the second elements of SP.

Running Time

• What's the running time of this algorithm?
  • During initialization, we visit each node once, which is O(n).
  • In the main loop, we remove each node from Est once, so there are O(n) repetitions of the main loop.
  • We look at each edge twice (one for each node connected to the edge), so the ``update distances'' part takes O(m) across the whole algorithm. (m is the number of edges)
  • How long does it take to find the smallest distance in Est? For the structures we've learned, O(n).
  • We've find the smallest distance O(n) times.
  • So, the running time is O(n^2 + m).
  • Since m is O(n^2), the running time is O(n^2).
• That's a significant improvement over the previous algorithm.
• Can we improve it more? In particular, can we speed up the determination of the smallest distance in Est?
  • We might consider using a Heap, since we're doing a lot of removeSmallest.
  • However, we do change priorities of objects, and Heaps aren't designed to easily support that operation. (How might you support a changePriority method?)

Some Other Graph Algorithms

Traveling Salesperson Problem

• The Traveling Salesperson Problem is, in effect, a variant of shortest path. Instead of finding the shortest path between two nodes, you find the shortest path (or cycle) that visits every node in the graph.
• Our ``brute force'' solution works here.
  • List all paths
  • Find the shortest
• That's an O(n!) algorithm
• Surprisingly, no one has found a substantially better algorithm (in terms of big-O analysis).
  • In fact, there are a class of problems with no known algorithms that run in time less than O(2ⁿ).

Minimum Spanning Tree

• An interesting variant of shortest path and traveling salesperson is the minimum spanning tree (MST) problem.
• The MST of an undirected graph is a set of edges that span the graph (permit one to get from each node to every other node).
  • It is minimum in the sense of having the smallest sum of edge weights.
• Note that the MST is a tree since there is no benefit to including a cycle (the extra edges can only add cost).
• It is possible to solve the MST problem by a greedy algorithm.
  • Actually, I've been told that there are a variety of greedy algorithms that solve the MST problem.
• Here's one version:
  • Separate the nodes into two groups: those in the MST and those not in the MST. Initially one node is in the MST (it doesn't matter which one).
  • Repeatedly pick the smallest edge between a node in the MST and a node not in the MST and add it to the MST.
  • If there are any nodes left in the set of nodes not in the MST, then there is no MST.

Topological Sort

• In a directed graph, we can treat the edges as giving an ordering.
• Unlike typical orderings, this isn't necessarily complete. That is, there can be unordered elements.
  • Even if there are edges from A to C and B to C, we still don't know anything about the relationship of A and B.
• In topological sort, you assign numbers to the elements of the graph in such a way that
  • No two elements have the same number
  • If there is a path from X to Y, then X has a lower number than Y.

History

• Created Monday, January 11, 1999.
• Added short summary on Friday, January 22, 1999.
• Filled in the details on Sunday, May 9, 1999. Many were based on outline 50 of CS152 98S.
• Added a few more notes on Monday, May 10, 1999.