The primary topics in this part of the specialization are: greedy algorithms (scheduling, minimum spanning trees, clustering, Huffman codes) and dynamic programming (knapsack, sequence alignment, optimal search trees).

Loading...

From the course by Stanford University

Greedy Algorithms, Minimum Spanning Trees, and Dynamic Programming

437 ratings

The primary topics in this part of the specialization are: greedy algorithms (scheduling, minimum spanning trees, clustering, Huffman codes) and dynamic programming (knapsack, sequence alignment, optimal search trees).

From the lesson

Week 1

Two motivating applications; selected review; introduction to greedy algorithms; a scheduling application; Prim's MST algorithm.

- Tim RoughgardenProfessor

Computer Science

Alright. So now that we've completed our warm up

Â by showing that at the very least, Prim's algorithm outputs a spanning tree. Let's

Â move on and actually show it outputs a minimum cost spanning tree.

Â And to prove this theorem, we're going to have to tackle head-on the kind of crisis

Â which you always face when designing a greedy algorithm.

Â So in a greedy algorithm, you're making an irrevocable decision,

Â like in Prim's algorithm, we're including an edge in our tree and never revisiting

Â it later. And, how can you be sure that you're not

Â making a mistake? How can you have a guarantee that the

Â decision you're making seemingly myopically right now is actually a good

Â decision won't come back to bite you later?

Â So it turns out for minimum spanning trees, there is a beautiful condition

Â which tells you when you're guaranteed to not regret including an edge into a

Â spanning tree, but guarantees when an edge has to belong

Â to the minimum spanning tree. So that's called the cut property,

Â it's the subject of the next slide. So this is a cool enough property that

Â we're going to bestow it not just with all caps but even with a box.

Â Now, that's a pretty good property. So what does it states?

Â Well, consider an edge of a graph, an edge that we are wondering if it's safe

Â to include it in the tree so far. So here is this sufficient condition

Â guaranteeing that you won't regret including this edge in the tree so far.

Â The condition is stated in terms of the cut.

Â So suppose you can find a cut, A, B with the property that amongst all edges in

Â the graph G, that happened across this cut, the edge E is the cheapest edge

Â crossing this cut. Okay? So not only should edge E cross

Â this cut, A, B, but it should cheapest such edge.

Â If this condition is met, then we definitely want them include the edge E

Â in our solution. Indeed, the edge E has to be a member of

Â any minimum spanning tree of the graph. So in this video, we're going to assume

Â that the cut property is true. It is by no means obvious.

Â It definitely requires a proof. I'll give you the proof in a separate

Â video. It's not,

Â it's a little bit tricky. It's based on a subtle exchange argument.

Â For this video, we're going to assume that it's true, however, and we just want

Â to be quiet, so that we want to figure out what it's good for.

Â Now, I will soon show you that it actually implies correctness of Prim's

Â algorithm. But just to get a feel for it, let's look

Â at a much simpler graph. Let's just look at a four cycle.

Â Four nodes, four edges with edge costs 1, 2, 3, and 4.

Â So let's look at, let's look at a few cuts.

Â So, let's look at the cut. We're on one side of the cut,

Â I put the upper right vertex, and on the other side of the cut, I put

Â the other three vertices. So there are two edges crossing this cut,

Â the edge that has cost 1, the edge that has cost 2.

Â So the edge with cost 1 is the cheapest edge crossing this cut, so by the cut

Â property, the edge of cost 1 has to be in the MST.

Â Okay. So we looked at one cut and both the cut

Â property [INAUDIBLE} to stick in the MST. That was pretty cool.

Â let's look another cut. Let's look at a cut where on one side, we

Â just put the bottom right vertex, and on the other side, we put all Now

Â this cut has two edges crossing it the edges that have cost 2 and cost 3.

Â The edge of cost 2 is the cheapest edge crossing this cut.

Â So by the cut property, it has to be in the MST.

Â So that's cool. So we know the two has got to be there.

Â Now let me point out something interesting that's happened,

Â which is that, it is not the case that this edge of cost 2 is the cheapest

Â crossing, every single cut that it crosses. Remember when we looked at cut

Â number 1, this edge of cost 2 was actually the most expensive edge crossing

Â that cut. But, we found a different cut that is the cheapest crossing and that's

Â enough to justify the cut property. So in other words,

Â all that's important for the cut property,

Â I just got to find you one cut for which an edge is the cheapest, that's enough to

Â conclude its presence in the MST. So similarly, we can look at a third cut

Â just consisting of the bottom left vertext and the other three vertices.

Â And it's the same story, there are two edges crossing this cut, the edge of cost

Â 3, the edge of cost 4. The edge of cost 3 is the cheapest edge

Â crossing this cut, so we know it's got to be in the MST.

Â And again, when we look at cut number 2, it didn't tell us whether or not the edge

Â of cost 3 is in the MST, but when we looked at cut number 3, that was enough

Â to conclude that the edge of cost 3 has to be in the MST. So there we go.

Â So we could use the cut property that construct an entire MST.

Â On the other hand, there's no way to use the cut property to try to justify the

Â edge of cost 4. Any cut that you pick for which the edge

Â of cost 4 crosses, there will be some other cheaper edge crossing it.

Â So you can never use the cut property as one would hope to justify the inclusion

Â of the edge of cost 4 and you'd better not be able to, because 4 is not in the

Â MST. Now a quick side note, some of you might

Â be wondering when I wrote in the conclusion of the cut property,

Â I said the MST of G, so that would seem to indicate that the minimum spanning

Â tree is unique. So that deserves a quick comment.

Â so first of all, if the edge costs are not distinct, if you have ties between

Â edges, then you can certainly have multiple different minimum spanning trees

Â and you have to state the cut property a little bit differently.

Â But again, in the lectures we are just going to assume distinct edge costs, so

Â that's not a problem. And in fact, something that will be a

Â consequence of the next slide, we'll notice that the minimum spanning tree is

Â unique with distinct edge cost. It's not obvious, but we'll prove it

Â shortly. All right.

Â So what I want to do to finish up this video is I want to assume that the cut

Â property is true. And then, from that, I want to derive, I

Â want to argue that Prim's algorithm is correct,

Â always outputs an MST. The proof of the cut property is

Â non-trivial and deserves its own video, which you can see separately.

Â All right. So given the tools that we've developed, this argument is actually

Â going to be quite short. So let's assume that the cut property is a true statement

Â and let's begin by building on the previous video.

Â The previous video argued that Prim's algorithm outputs a spanning tree, didn't

Â argue it was a minimum one, but it argued it's a spanning tree, it spans all the

Â vertices and has no cycles. Let's call the output of Prim's algorithm

Â at the end of algorithm T star. Now,

Â stare at the cut property, stare at the pseudocode of Prim's

Â algorithm. What happens in each iteration of Prim's

Â algorithm? Well, we have our set capital X, that's

Â what we spanned so far. There's the rest of this stuff, V - X, so

Â that's a cut X, V - X. What does Prim choose to include next?

Â Well, in brute-force searches through the edges, the cross is cut and it adds the

Â cheapest one of them. Well, that is right in the wheelhouse of

Â the cut property. What does the cut property says? It says

Â cheapest edges crossing cuts have to be in the MST.

Â So they just fit together beautifully. Prim's algorithm explicitly picks an edge

Â at each iteration which satisfies the hypothesis of the cut property and

Â therefore has to be in the MST. So remember, the conclusion of the cut

Â property says edges so justified must belong to the MST. So if everything in T

Â star is justified by the cut property, then everything in T star is in the MST

Â so T star is a subset of the MST. But T star, of course, as we have argued,

Â is already a spanning tree in it of itself.

Â And, if you add more edges to T star, it's no longer going to be a spanning

Â tree, because you are going to pick up cycles, right? If you ever have something

Â that is connected, there is a path from each pair of vertices, and you add a new

Â edge, you are going to close a path, you're going to get a loop.

Â Okay? So T star is already a spanning tree, and

Â you can't have anything bigger and still be a spanning tree.

Â So therefore, this has to be the minimum spanning tree,

Â there cannot be anything else. So for this reason, T star must in fact

Â be the minimum cost spanning tree of the graph.

Â Since the input graph was arbitrary, assuming only it was connected, this

Â completes, assuming the cut property, the proof of correctness of Prim's minimum

Â spanning tree algortihm.

Â Coursera provides universal access to the worldâ€™s best education,
partnering with top universities and organizations to offer courses online.