0:00

In this video, we'll be talking about

Â simple linear regression and multiple linear regression.

Â Linear regression will refer to one independent variable to make a prediction.

Â Multiple linear regression will refer to

Â multiple independent variables to make a prediction.

Â Simple linear regression or SLR is

Â a method to help us understand the relationship between two variables.

Â The predictor independent variable x and the target dependent variable y.

Â We would like to come up with a linear relationship between the variables shown here.

Â The parameter b_0 is the intercept.

Â The parameter b_1 is the slope.

Â When we fit or train the model,

Â we will come up with these parameters.

Â This step requires lots of math,

Â so we will not focus on this part.

Â Let's clarify the prediction step.

Â It's hard to figure out how much a car costs,

Â but the highway miles per gallon is in the owner's manual.

Â If we assume there is a linear relationship between these variables,

Â we can use this relationship to formulate a model to determine the price of the car.

Â If the highway miles per gallon is 20,

Â we can input this value into the model to obtain a prediction of $22,000.

Â In order to determine the line,

Â we take data points from our data set marked in red here.

Â We then use these training points to fit our model.

Â The results of the training points are the parameters.

Â We usually store the data points into data frame or numpy arrays.

Â The value we would like to predict is called the target that we store in the array y.

Â We store the dependent variable in the data frame or array x.

Â Each sample corresponds to a different row in each data frame or array.

Â In many cases, many factors influence how much people pay for a car.

Â For example, make or how old the car is.

Â In this model, this uncertainty is taken into account by

Â assuming a small random value is added to the point on the line.

Â This is called noise.

Â The figure on the left shows the distribution of the noise.

Â The vertical axis shows the value added and

Â the horizontal axis illustrates the probability that the value will be added.

Â Usually a small positive value is added or a small negative value.

Â Sometimes, large values are added.

Â But for the most part,

Â the values added are near zero.

Â 2:32

We can summarize the process like this.

Â We have a set of training points.

Â We use these training points to fit or train the model and get parameters.

Â We then use these parameters in the model. We now have a model.

Â We use the hat on the y to denote the model is an estimate.

Â We can use this model to predict values that we haven't seen.

Â For example, we have no car with 20 highway miles per gallon.

Â We can use our model to make a prediction for the price of this car.

Â But don't forget our model is not always correct.

Â We can see this by comparing the predicted value to the actual value.

Â We have a sample for 10 highway miles per

Â gallon but the predictive value does not match the actual value.

Â If the linear assumption is correct,

Â this error is due to the noise.

Â But there can be other reasons.

Â To fit the model in Python,

Â first we import linear model from

Â sklearn then create a linear regression object using the constructor.

Â We define the predictor variable and target variable then use the method

Â fit to fit the model and find the parameters b_0 and b_1.

Â The input the features and the targets.

Â We can obtain prediction using the method predict.

Â The output is an array.

Â The array has the same number of samples as the input x.

Â The intercept b_0 is an attribute of the object lm.

Â The slope b_1 is also an attribute of the object lm.

Â The relationship between price and highway miles per

Â gallon is given by this equation in bold,

Â price equals 38,423.31 minus

Â 821.73 times highway miles per gallon,

Â like the equation we discussed before.

Â 4:43

If we have for example 4 predictor variables then b_0

Â intercept x equal zero b _1 the coefficient or parameter of x_1,

Â b_2 the coefficient of parameter x_2 and so on.

Â If there are only two variables then we can visualize the values.

Â Consider the following function.

Â The variables x_1 and x_2 can be visualized on a 2D plane.

Â Let's do an example on the next slide.

Â The table contains different values of the predictor variables x_1 and x_2.

Â The position of each point is placed on the 2D plane color-coded accordingly.

Â Each value of the predictor variables x_1 and x_2 will be mapped to a new value y, y hat.

Â The new values of y y hat are mapped in

Â the vertical direction with height proportional to the value that y hat takes.

Â We can fit the multiple linear regression as follows.

Â We can extract the four predictor variables and store

Â them in the variable z then train the model

Â as before using the method train with

Â the features or dependent variables and the targets color.

Â We can also obtain a prediction using the method predict.

Â In this case, the input is an array or data frame with four columns.

Â The number of rows corresponds to the number of samples.

Â The output is an array with the same number of elements as number of samples.

Â The intercept is an attribute of the object and the coefficients are also attributes.

Â It is helpful to visualize the equation,

Â replacing the dependent variable names with actual names.

Â This is identical to the form we discussed earlier.

Â