Welcome back. In the last session we have defined,

we have constructed the input and output data as extreme and widely.

Now we are coming to the most interesting topic which is designing of the LSTM network.

But first of all,

we need to import some required packages.

So, what we are importing here,

we're importing here dense class,

we're importing here input layer,

LSTM layer, and we're importing here the model class.

I also have here import h5py package

because I use it to store the trained LSTM model.

So, let's start. The first class which we have here is the input layer.

We're using here functional API of Keras.

There are two flavors of how we can designed neural networks with Keras,

one is sequential API.

This is the most known actually.

And another one is the functional API which is much more flexible.

So, I have decided to start to learn immediately with functional API because later on,

you will see that you can do much easier more complicated things with functional API.

So, first layer is the input layer.

As always every neural network needs an input.

It takes the argument batch shape.

This is a three dimensional arrays.

So, it consist of batch size which we have defined as 64,

it has timesteps which are 10,

and it has number of parameters.

So, we are dealing only with

crude oil prices and this is only one dimension which we have.

So, we are predicting the crude oil price is one dimension,

so we have here only one.

So, suppose we would predict crude oil price for Europe,

crude oil price for US at the same timesteps,

so we would have here two parameters but we have only one.

And the output of

this class we are storing in the input is one parameter.

Secondly, we are defining the LSTM layer.

This LSTM layer takes its arguments several parameters.

So actually, if you look at Keras documentation,

you will see that it has a lot of parameters

but we are dealing only with the most important ones.

And here the first one is the number of LSTM notes.

So, if I have here only one node,

we can see the whole layer is a one node,

is a one memory cell.

This is what I have spoken about in the previous session.

I said that LSTM layer is roughly equal to one node,

to one LSTM cell.

This is really very roughly,

and I have said it not to confuse you with too much information at that point of time.

But now, we know LSTM layer can take much more nodes,

you can say also neuron's.

So, here I have decided to

use 10 nodes in the both layers.

So, we see 10 nodes,

then we see stateful true.

So, we have decided that our LSTM network will be stateful therefore,

we are setting it's true.

So, the second one is the return sequences,

it's a separate point.

And we're setting here return sequences, true.

That means that it will return the whole sequence,

the whole output sequence for every timesteps.

So, we have here 10 timesteps,

it will return a sequence of 10 outputs.

And this one, the output of this layer will be the input of the next layer which is,

actually this is architecture of stacked LSTM layers.

And you'll see that the output of this LSTM layer is stored here, lstm_1_mae.

And it goes as input into the next layer lstm_1_mae here.

And if you look carefully on the whole structure,

you'll see that the input layer is stored here in this variable, inputs_1_mae.

This output of this layer goes as input into the next layer here.

So, then we have output of the LSTM layer which is stored here lstm_1_mae,

and it goes as input into the next LSTM layer.

The output of this layer is lst_2_mae,

so now we have the structure.

And now, we need a dense layer,

dense actually is a fully connected layer.

As you know from another neural networks which are actually

consisting only from such layers like multilayer perceptron,

this is a typical picture of neural network they consisting of

dense layers and we need this layer to output our data.

So, it takes one argument which is units.

Units Is again, one.

This is the same as one here.

This is only one dimension.

We are dealing only with one thing which we are going to

predict which is the crude oil price, one unit.

And this class takes its input lstm_2_mae from here,

and we have the output.

So, now we are defining the model.

The model class takes two arguments,

the inputs which is the input layer and the outputs which is the output layer,

this is the output_1_mae.

And this is already our regressor.

So, if we define the model we have our regressor,

we have our model.

In the last step we have to compile this model.

So, meaning that the whole thing,

the whole structure will be built-up and connected.

So we call this compile method,

everything what we described here is going to be designed and connected.

So, we use this optimizer and the compile method.

We're using optimizer which is Adam,

and we use loss function which is mean absolute error.

In the next sessions, we are going to see that we can use also another function,

another loss function which is called MSE, means squared error.

And we are going to compare which loss function performs better,

mean absolute error or mean squared error,

and what are the differences.

So, now we can execute this.

And we see this is our LSTM layer.

It has input layer,

it has two LSTM layer,

and it has dense layer.

In the next session,

we're going to dive deeper into the structure and we're going to

look onto anatomy of the LSTM layer.

And until the next time,

stay tuned, enjoy. Bye.