0:00

Recalling the last lecture, that when we had an outcome, y, and the regressor x,

Â y was n by 1, and x was the regressor that was n by 1.

Â 0:17

our minimum worked out to be beta hat = the inner product of y and

Â x, over the inner product of x with itself.

Â Now, imagine if we we're to look at the y and

Â x pairs, this fits a line to the origin.

Â However, let's add some data over here.

Â A line to the origin may or may not make sense.

Â So, let me draw a picture that's a little bit closer to what I'm thinking.

Â So, imagine if you had a setting like this.

Â There's a clear linear relationship, but

Â trying to fit a line to the origin to that data set isn't going to work so well.

Â 1:03

So what you might consider doing is resetting the origin

Â to somewhere more relevant, okay?

Â Or of course fitting a line that also has an intercept.

Â But let's imagine if all you had at your disposal was regression to the origin.

Â First thing you might wanna do is just reset the origin

Â to be right in the middle of the data.

Â Then when you were to fit a line to the origin it

Â would go right through the data in a pretty reasonable fashion.

Â At least it would take care of the problem that

Â the origin is sort of nowhere near the data, and it's forcing the line to

Â 1:43

So how would we get right in the middle of the data?

Â Well, we would just say, let's define y tilde there as the centered version of y,

Â one where we've subtracted off the mean from every data point, so

Â that it now has mean 0.

Â And then this is ostensibly just shifting the origin right into the middle

Â of the dataset.

Â So, that would just be (I- Jn(Jn

Â transposed Jn inverse) Jn transpose)y.

Â And then, if we were to center our x, that's of course I minus the same matrix

Â 2:28

Times x.

Â So now if we were to do regression through the origin of these two matrices,

Â I try to minimize y tilde- x tilde beta

Â squared, the norm of that squared,

Â what would be the beta hat, in fact just to differentiate it from the beta before,

Â let's call that gamma, what would be the gamma hat that we would get?

Â Well, it's of course, gamma hat would be norm,

Â I'm sorry, in a product of y tilde and

Â x tilde divided by the inner product of x tilde by itself, okay?

Â Well that is equal to y transpose times (I minus, and

Â let me just replace this matrix so I don't have to keep writing it.

Â Let me just call it H, (I- H)

Â transpose times (I- H) then times x,

Â all over x transposed times (I- H) (I- H)x.

Â We'll transpose there.

Â So now if you go back to our previous lecture, or

Â a couple lectures previous, when we we're talking about variances,

Â what you can see is that this works out to be this quantity in the top.

Â 4:06

And this quantity in the denominator is (n- 1) var (x).

Â Well, we can manipulate that.

Â And let's, to make it a little bit more statistical,

Â let's write as rho hat xy is the empirical correlation between y and x.

Â And sigma squared x hat is the empirical

Â variance of x in sigma hat squared y is the empirical variance of y,

Â okay, cuz that makes it seems a little more statistical.

Â This language over here seems more like we're writing computer code than writing

Â mathematical notation.

Â Okay, so let's take that.

Â So the covariance is nothing other than, we can write the numerator, right?

Â The numerator is (n- 1).

Â That could be written as the correlation between y and

Â x times the standard deviation for x and a standard deviation for y,

Â and the denominators of course, the variance of x.

Â 5:40

And that's basically saying that the slope, if we were to center our y regress,

Â our outcome, and center our regressor, and fit the regression to the origin,

Â the slope of the best fitting regression line is the correlation between the y and

Â the x, the estimated correlation between the y and

Â the x times the ratio of the standard deviations.

Â Now a couple things to note.

Â First of all, the units of this quantity are correct.

Â So the slope has to be in units change in y over change in x.

Â So units of y divided by units of x.

Â Okay, but let's look at this quantity.

Â The correlation is a unit for

Â e quantity, then it's multiplied by the standard deviation of the y,

Â so that's the units of y divided by the standard deviation of the x.

Â So that's the units of x.

Â So this quantity, our estimated slope has units

Â 6:33

y units divided by x units, which is what it has to have.

Â It's also interesting that if we reverse the relationship, and fit x as the outcome

Â and y as the predictor, all that happens then is these two reverse themselves,

Â because the correlation, it doesn't matter which argument is first.

Â Then it works out to be rho hat xy, sigma hat x divided by sigma hat y.

Â So, that also implies, you'll notice what this also implies is if we

Â standardize our two regression variables in addition to centering them

Â before we do regression to the origin, then both their variances are 1.

Â And the regression to the mean, I'm sorry the regression,

Â correlation works out to just regression,

Â slope estimate works out to just be the correlation, okay?

Â So that's maybe everything you need to know about regression to the origin.

Â And the big take home message is that if you center your variables first,

Â regression to the origin leads to, and

Â we'll see in a minute this leads to the exact same

Â regression slope than if we fit a line that has both an intercept and a slope.

Â But it works out to be the correlation between the x's and

Â the y's times the ratio of the standard deviations.

Â Okay, so let's try some computer code to just illustrate this and

Â compare it with what LMRS function for regression is doing.

Â