0:00

Let's talk a little bit about the normal distribution.

Â So the standard normal density, a random variable follows

Â a standard normal, if it's associated population density

Â is 1 / square root 2 pi and then e to the -z squared / 2.

Â For z between minus infinity and plus infinity.

Â 0:24

And then the non-standard normal distribution is any

Â random variable x that has the distribution of mu plus sigma z,

Â where z is a standard normal and the density of that random

Â variable works out to be phi of x- mu / sigma divided by sigma.

Â So we can write out the non-standard normal density as a simple function of

Â the standard normal density.

Â And the non-standard normal density looks to be like 1 over square

Â root 2 pi sigma squared e to the negative ( x- mu ) squared over 2 sigma squared.

Â And again, from minus infinity less than x less than plus infinity.

Â 1:12

For the standard normal distribution, all of the odd moments are zeroes.

Â So expected value of z, expected value

Â of z cubed, and so on, are all 0.

Â 1:30

The normal distribution, the non-standard normal distribution is

Â characterized by the mean and the variance.

Â So we might write if something is non standard normal distribution just,

Â a normal distribution.

Â We might write that x is normal mu sigma squared.

Â Okay, so hopefully all of this is review for you.

Â And we can use this to build up the multivariate normal distribution.

Â So we're going to say that a vector, z, let's say is

Â a standard multivariate normal if its density satisfies,

Â 2:25

So this is just by the way equal to the product

Â i = 1 to n of 2 pi to the minus 1/2,

Â e to the negative zi squared over 2.

Â Okay, so, we get the multivariate normal distribution just as the product of

Â a collection, the multivariate standard normal distribution just is the product

Â of a collection of IID standard normals.

Â So that's for an n by 1 vector z.

Â And then we might define a multivariate non-standard normal, say x.

Â So it would not necessarily mean zero in variance/covariance matrix is I.

Â We might define that as mu plus sigma to the 1/2 times z.

Â Where sigma is a variance/covariance matrix, sigma to the 1/2 matrix,

Â that decomposition is called the Cholesky decomposition.

Â So notice if we write x this way, the expected value of x is equal to mu,

Â because the expected value of z is 0, and

Â the variance of x is equal to sigma to the 1/2 variance of z,

Â variance of z, sigma to the 1/2 transpose.

Â Which variance of z is I, so we just use this fact and it equals sigma.

Â Okay, so using that we can define this non-standard normal,

Â multivariate normal distribution.

Â Which people would then just call then the normal distribution or

Â the multivariate normal distribution maybe.

Â And we would write that x is normal, mu, sigma.

Â Now, if the density is associated with it,

Â we could use the transformation to figure out the density associated with it.

Â And it is 2 pi, to the -n/2,

Â determinant of sigma to the -1/2,

Â e to negative x minus mu transpose sigma

Â inverse x minus mu divided by 2.

Â 4:48

Okay, so that's what the non-standard normal,

Â what we would just call the normal distribution works out to be.

Â The normal distribution,

Â the multivariate normal distribution has many convenient properties.

Â First of all, any subvector of x, any subvector of x also follows a multivariate

Â normal distribution with of course the relevant mean and variant.

Â So for example if I take x and break it into two column vectors x1 and

Â x2 and my mu has the same corresponding column vectors,

Â mu1, mu2 and my sigma is now a matrix with,

Â 5:46

And similarly, x2 is going to be normal with mean mu and variance of sigma22.

Â Should have been sigma11.

Â Okay, so any subvector of a multivariate normal vector is multivariate normal.

Â The second thing that is true about the multivariate normal is any full

Â rank linear combination of multivariate normals is multivariate normal.

Â So let's suppose I take Ax + b.

Â Then we know what the expected value of that is.

Â The expected value of Ax + b has to be equal to A mu + b.

Â And we know what the variance of Ax + b has to be,

Â 6:29

That has to be A variance of x which is sigma, A transpose.

Â However, we also know when x is multivariate normal

Â that then the variable Ax + b, let's call that w.

Â Then we know that w has to be normally distributed

Â also with mean A mu + b and variance A sigma A transpose.

Â So the normal distribution has this property that all linear,

Â the multivariate normal distribution has this property that all full ranked linear

Â combinations of multivariate normals are also multivariate normal.

Â In addition, all conditional distributions, so

Â if I take, for example, x1 given x2 where x1 and

Â x2 are defined as above, that will also be normally distributed.

Â So all linear combinations,

Â all marginals, and all conditional distributions of

Â the multivariate normal distribution are also multivariate normal.

Â Another property of the multivariate normal is that absence of covariance

Â implies independence.

Â So take for example my x1 and x2 up here and I look at its covariance matrix,

Â I have it written out right here.

Â It's sigma12, which is I wrote sigma21 there, but

Â that's just sigma12 transpose because the matrix has to be symmetric.

Â If that's 0, so in other words if the block off diagonal of the sigma matrix,

Â sigma matrix is 0 then that would imply that x1 is not just uncorrelated with x2,

Â but it's actually independent of x2, okay?

Â So the normal distribution is one of the strange distributions where this is true.

Â Where not only does independence imply absence of covariance which is true for

Â all distributions, but the reverse direction also applies.

Â Absence of covariance implies independence.

Â So in many ways, the multivariate normal distribution is probably the most

Â convenient distribution to work with.

Â