Taming a beast

Loading...

From the course by Imperial College London

Mathematics for Machine Learning: Multivariate Calculus

1116 ratings

Imperial College London

Mathematics for Machine Learning: Multivariate Calculus

1116 ratings

Course 2 of 3 in the Specialization Mathematics for Machine Learning

This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future.

From the lesson

What is calculus?

Understanding calculus is central to understanding machine learning! You can think of calculus as simply a set of tools for analysing the relationship between functions and their inputs. Typically, in machine learning, we are trying to find the inputs which enable a function to best match the data. We start this module from the basics, by recalling what a function is and where we might encounter one. Following this, we talk about the how, when sketching a function on a graph, the slope describes the rate of change off the output with respect to an input. Using this visual intuition we next derive a robust mathematical definition of a derivative, which we then use to differentiate some interesting functions. Finally, by studying a few examples, we develop four handy time saving rules that enable us to speed up differentiation for many common scenarios.

Taming a beast5:38

See you next module!0:39

Meet the Instructors

Samuel J. Cooper
Lecturer
Dyson School of Design Engineering
David Dye
Professor of Metallurgy
Department of Materials
A. Freddie Page
Strategic Teaching Fellow
Dyson School of Design Engineering

In this video, we're going to work through a nasty looking function that will

require us to use all four of the time-saving rules that we've learned so far.

To make matters worse, this function isn't going to describe anything familiar.

So, we'll just be flying blind and have to trust the maths.

This will hopefully give you the confidence to

dive into questions in the following exercise.

Consider the rather nasty function f of x equals sin of

2x to the power of five plus 3x all over e to the 7x.

The essence of the sum product in the chain rules are all about

breaking your function down into manageable pieces.

So, the first thing to spot is that although it is currently expressed as a fraction,

we can rewrite f of x as a product by moving

the denominator up and raising it to the power of minus one.

There is actually another rule specifically

for dealing with fractions directly called the quotient rule,

but it requires memorizing an extra expression.

So we're not going to cover it in this course as you can always use

the approach that we've used here and rewrite your friction as a product.

Next, we'll split f of x up into the two parts of

the product and work out how to differentiate each parts separately,

ready to apply the product rule later on.

Let's start with the first part,

which we can call g of x.

We've got the trigonometric functions sin applied to a polynomial 2x to the 5 plus 3x.

This is a classic target for the chain rule.

So, all we need to do is take our function and

split up into the two parts in order to apply the chain rule.

So, we can take g of u equal sin of u,

and u of x equals 2x to the power five plus 3x.

Now, we've got these two separate functions,

and we're going to want to differentiate each of them in order to apply the chain rule.

So we say, okay,

g dash of u is going to just equal sin differentiates to cos.

So, cos u, and u dash of x is

going to equal 10x to the power of four plus three.

Now we've got these two expressions,

and I'm going to use a mix of notation for convenience.

And I'm going to say, okay, well,

dg by du multiplied

by du by dx is going to give us,

so we've got cos of u multiply by 10x to the power of four plus three.

And now we're going to want to have a final expression that doesn't include u.

So we can think about these two cancel each other out,

and we get dg by dx.

And this just equals k cos of u,

which is just 2x to the power of

5 plus 3x multiplied by 10x to the power of four plus three.

And that's it. We now have an expression for

the derivative g of x and already we've made use of the chain rule,

the sum rule, and the power rule.

Now, for the second half of our original expression,

which we will call h of x,

once again, we can simply apply the chain rule after splitting our function up.

So we can say that h of v equals e to the v,

and v of x equals minus 7x.

Now, once again, we've got our two functions.

We just want to find the derivatives.

So, h prime of v is just going to equal, well we've seen this one before, e to the v again.

And v prime of x is going to just be minus seven.

So, combining these two back together and using different notation,

we can say that dh by dv multiplied by dv by dx is just going to give us,

we've got minus seven times e to the minus 7x.

And actually, already all of our v's have disappeared.

So, this is already our final expression.

As we now have expressions for the derivatives of both parts of our product,

we can just apply the product rule to generate the final answer, and that's it.

We could rearrange and factorize this in

various fancy ways or even express it in terms of the original function.

However, there is a saying amongst coders that I think can be applied quite generally,

which is that premature optimisation is the root of all evil, which in this case means,

don't spend time tidying things up and rearranging

them until that you're sure that you've finished making a mess.

I hope that you've managed to follow along with this example,

and will now have the confidence to apply our four rules to other problems yourself.

In calculus, it's often the case that some initially scary looking functions,

like the ones we worked with just now,

turn out to be easy to tame if you have the right tools.

Meanwhile, other seemingly simple functions occasionally turn out to be beasts,

but this can also be fun.

So, happy hunting.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play