So we found out about Grad and that led us on to sketch out a neat method for

finding the minima or maxima of multivariable functions which we called gradient descent.

In this video, we'll look at what happens if we want to find the minima or maxima subject

to some constraint that we want to

find the maxima somewhere along a line or something like that.

This is called a method using Lagrange multipliers.

So what we did in the last video is we looked at a function like this one f here.

This function f is x times the exponential of minus x

squared plus y squared and it's one of the standard Matlab examples.

I plotted below here a contour map of the valleys.

If I now plot just the contour map,

I can plot out the vectors of grad f. As we said,

these vectors are perpendicular to

the contour lines and they point up the steepest gradient.

Now let's return to the function we were looking at last time,

f equals x squared y which is the Khan Academy standard example for this problem.

Last time we plotted grad f on

both the 3D version and then drop that down onto the corresponding contour map.

Actually is going to be easier if we now just work with the contour map itself.

So here's the contour map and what I've done is,

I've used the gradient function in Matlab to just plot out

the gradient perpendicular to contours everywhere in the space.

So the low sides are at negative y down here and the high sides are at positive y up here.

Now what happens if I want the maximum value for this function f equals x squared y,

but constrain it to a lie on a circle.

Where is the highest point anywhere on that circle?

Say I have a circle of radius a squared and I want to find

the minima and maxima on that path as I go around that circle.

I can imagine that describing an equation for how

x squared y varies as I go round a circular path,

is going to be a complete nightmare.

But that's not the question we actually asked.

What we wanted to know was,

what's the maximum along that circle?

Not what's the value everywhere on the circle.

Now Lagrange was one of those French mathematicians whose name is inscribed around

the Eiffel Tower and what Lagrange

noticed was that when the contours just touch the path,

then you'll have found the maximum or minimum point. That is,

when f is a little bit smaller,

it won't quite touch the path.

See here in 3D. And when it's a bit bigger it might cross a couple of times.

See here in 2D on the contours.

But when the contour just touches,

then we'll have found the minima and the maxima of

the function f as we go round the path, our circle in this case.

And what Lagrange noticed was that

when the contour touches the path then the vector perpendicular to the contour is in

the same direction up to

a minus sign as the vector of the path itself that's perpendicular to path.

So if we can find Grad,

we can find the minimum maximum points to solve the problem.

If we can find Grad perpendicular to the contour

on both the path and the function we're away.

So we want to maximise the function f of x y which is equal to x squared y,

subject to some constraint which we're going to call the constraint equation G of x y.

And that's the equation of a circle x squared plus y squared and it has some value,

some particular value to the problem we want to solve a squared.

So what we're saying is,

if we've got the function doing something like this and it's grad that

way and the circle has

some called path it's doing that and it's got it's grad that way.

What we're doing is we're solving Grad f is equal to

lambda some number times Grad g where lambda is the Lagrange multiplier.

And that's all we need to do.

We just need to set up this set of equations and then solve them.

So, if I take Grad f is equal to the Grad of x squared y.

Well that's equal to, if I differentiate for the df/dx,

I've got 2xy and if I take df/dy the y just goes and I've got x squared.

And I'm saying that's equal to lambda times Grad g,

which is equal to lambda times,

if I differentiate g x squared plus y square with respect to x I've got 2 x.

If I differentiate with respect to y, I've got 2y.

And that's giving me two equations and two unknowns and I've got

a third equation which is the constraint equation itself that

brings in the actual value of the circle I'm particularly interested in.

So I've just got to solve that.

So if I take the first line,

I've got 2xy is equal to lambda times 2x.

I cancel the two x's and therefore I've got y is equal to lambda straight out.

If I take the second row,

the y row if you like,

I've got x squared is equal to lambda times 2y.

But lambda is itself y so that's equal to 2y squared.

So I can say that x is equal to say the square root of two times y.

Because I've done the square root I've got a plus minus here.

And now if I take the constraint equation itself, the third one,

I've got x squared plus y squared is

equal to a squared but x squared is equal to two y squared.

So that's equal to three y squared.

So then if I square root that I can say that y is equal

to a divided by the square root of three.

And again because I've done the square root,

I've got a plus minus.

So that is my solutions.

So I can write those out.

And I've got the solutions are going to be a over the square over

three times if I take y is one then x is root two times that, so root two one.

And I've got a over root three times root two minus one for

y. I've got all the possibilities a over root three minus root two and one.

I've got a over the square root three minus root two and minus one.

Now, if I find the values of the function f of (x,y),

as I go take all of those.

I've got to find x squared y for that.

So that's a cubed

over three root three because I'm cubing that in effect because they're both in x and y.

And I've got x squared is two,

times y is one.

So that two.

Here, I've got the same thing but I've got what y is negative.

So now I've got a cubed over three root three times minus two.

This one's y is plus added when they squares the minus is going to disappear.

So this is going to be another plus solution.

Two over three root three a cubed.

And I've got here,

here why is negative so I'm going to get a minus here,

a cubed over three root three.

So I've got a max,

I've got a min,

I've got a max and another min.

So I've got two positive solutions and I wanted to find the maximums.

So that those are there and I've got the minima as well f.

So let's look at what that looks like now on the graph.

So two of our solutions are here, two here.

When we switch to the 3D view we can see that the two with

positive y are maxima and the two with negative y are the minima.

So that's really neat. What we've done

here is we've used our understanding of gradients to

find the minima or maxima subject to

some constraint equation like a straight line or a circle.

And very often where we will want to make it but some other variables at

a function that we want to fit are fixed in relation to each other.

They have some fixed relationship like they were a circle.

So this is a very handy thing to be able to do.

It's really useful. So hopefully that was good fun.