Welcome to the fourth week of this course.

By now, you've seen four promulgation and back promulgation in the context

of a neural network, with a single hidden layer, as well as logistic regression, and

you've learned about vectorization, and

when it's important to initialize the ways randomly.

If you've done the past couple weeks homework, you've also implemented and

seen some of these ideas work for yourself.

So by now,

you've actually seen most of the ideas you need to implement a deep neural network.

What we're going to do this week, is take those ideas and put them together so

that you'll be able to implement your own deep neural network.

Because this week's problem exercise is longer,

it just has been more work, I'm going to keep the videos for

this week shorter as you can get through the videos a little bit more quickly, and

then have more time to do a significant problem exercise at then end, which I hope

will leave you having thoughts deep in neural network, that if you feel proud of.

So what is a deep neural network?

You've seen this picture for logistic regression and

you've also seen neural networks with a single hidden layer.

So here's an example of a neural network with two hidden layers and

a neural network with 5 hidden layers.

We say that logistic regression is a very "shallow" model,

whereas this model here is a much deeper model, and

shallow versus depth is a matter of degree.

So neural network of a single hidden layer,

this would be a 2 layer neural network.

Remember when we count layers in a neural network, we don't count the input layer,

we just count the hidden layers as was the output layer.

So, this would be a 2 layer neural network is still quite shallow,

but not as shallow as logistic regression.

Technically logistic regression is a one layer neural network,

we could then, but over the last several years the AI,

on the machine learning community, has realized that there are functions that

very deep neural networks can learn that shallower models are often unable to.

Although for any given problem, it might be hard to predict in advance exactly how

deep in your network you would want.

So it would be reasonable to try logistic regression, try one and

then two hidden layers, and view the number of hidden layers as another hyper

parameter that you could try a variety of values of, and

evaluate on all that across validation data, or on your development set.

See more about that later as well.

Let's now go through the notation we used to describe deep neural networks.

Here's is a one, two, three, four layer neural network,