0:00

in the earlier videos from this week as

Â well as from the videos from the past

Â several weeks you've already seen the

Â basic building blocks of board

Â propagation and back propagation the key

Â components you need to implement a deep

Â neural network let's see how you can put

Â these components together to build a

Â deep net use the network with a few

Â layers let's pick one layer and look at

Â the computations focusing on just that

Â layer for now so for layer L you have

Â some parameters WL and Bo and for the

Â forward prop you will input the

Â activations a L minus one from the

Â previous layer and output Al so the way

Â we did this previously was you compute Z

Â l equals WL x al minus one plus BL um

Â and then al equals G of Z L right so

Â that's how you go from the input al

Â minus one to the output Al and it turns

Â out that for later use will be useful to

Â also cache the value ZL so let me

Â include this on cache as well because

Â storing the value Z L will be useful for

Â backward for the back propagation step

Â later and then for the backward step or

Â three for the back propagation step

Â again focusing on computation for this

Â layer L you're going to implement a

Â function that inputs da of L and outputs

Â da L minus one and just a special the

Â details the input is actually da FL as

Â well as the cache so you have available

Â to you the value of ZL that you compute

Â it and

Â in addition to outputting GL minus 1 you

Â will output

Â you know the gradients you want in order

Â to implement gradient descent for

Â learning ok so this is the basic

Â structure of how you implement this

Â forward step I'm going to call it a

Â forward function as well as backward

Â step we shall call it back wave function

Â so just to summarize in layer L you're

Â going to have you know the forward step

Â or the forward property' forward

Â function input a L minus 1 and output al

Â and in order to make this computation

Â you need to use WL n PL um and also

Â output a cache which contains ZL and

Â then on the backward function using the

Â back prop step will be another function

Â then now inputs the AFL and outputs da

Â ll minus 1 so it tells you given the

Â derivatives respect to these activations

Â that's da FL what are the derivatives or

Â how much do I wish you know a L minus 1

Â changes computed derivatives respect to

Â D activations from the previous layer

Â within this box right you need to use WL

Â and BL and it turns out along the way

Â you end up computing DZ l um and then

Â this box this backward function can also

Â output DW l and DB l well now sometimes

Â using red arrows to denote the backward

Â generations so if you prefer we could

Â draw these arrows in red so if you can

Â implement these two functions then the

Â basic computation of the mirror network

Â will be as follows you're going to take

Â the input features a 0 see that in and

Â that will compute the activations of the

Â first layer let's call that a 1 and to

Â do that you needed W 1 and B 1 and then

Â we'll also you know cache

Â the way z1 now having done that you feed

Â that this is the second layer and then

Â using W 2 and B 2 you're going to

Â compute the activations our next layer a

Â 2 and so on until eventually you end up

Â outputting a capital L which is equal to

Â Y hat and along the way we cashed all of

Â these on values Z so that's the forward

Â propagation step now for the back

Â propagation step what we're going to do

Â will be a backward sequence of

Â iterations in which you're going

Â backwards and computing gradients like

Â so so as you're going to feed in here da

Â L and then this box will give us da of L

Â minus 1 and so on until we get da - da 1

Â you could actually get one more output

Â to compute da 0 but this is derivative

Â respect your input features which is not

Â useful at least for training the weights

Â of these are supervised neural networks

Â so you could just stop it there belong

Â the way back prop also ends up

Â outputting DW l DB l right this used

Â upon so wo and BL um this would output d

Â w3 g p3 and so on so you enter

Â computing all the derivatives you need

Â 6:27

WL PL and it turns out that we'll see

Â later that inside these boxes we'll end

Â up computing disease as well so one

Â innovation of training for a new network

Â involves starting with a zero which is X

Â and going through for profit as follows

Â computing Y hat and then using that to

Â compute this and then back prop right

Â doing that and now you have all these

Â derivative terms and so you know W will

Â get updated as some W minus the learning

Â rate times DW right for each of the

Â layers and similarly for B right now

Â we've compute the back prop and have all

Â these derivatives so that's one

Â iteration of gradient descent for your

Â neural network now before moving on just

Â one more implementational detail

Â conceptually will be useful to think of

Â the cashier as storing the value of Z

Â for the backward functions but when you

Â implement this you see this in the

Â programming exercise when you implement

Â it

Â you find that the cash may be a

Â convenient way to get the value of the

Â parameters at W 1 V 1 into the backward

Â function as well so the program exercise

Â you actually spawn the cash is Z as well

Â as W and B all right so to store z2w to

Â be to go from an implementational

Â standpoint I just find this a convenient

Â way to just you know get the parameters

Â copied to where you need to need to use

Â them later when you're computing back

Â propagation so that's just an

Â implementational detail that you see

Â when you do the programming exercise so

Â you've now seen one of the basic

Â building blocks for implementing a deep

Â neural network image layer there's a for

Â propagation step and there's a

Â corresponding backward propagation step

Â and as

Â cash deposit information from one to the

Â other in the next video we'll talk about

Â how you can actually implement these

Â building blocks let's go into the next

Â video

Â