Simple Linear Regression

… a core concept used in Quantitative Methods and Atlas104

Concept description

Wikipedia (reference below) defines simple linear regression as a linear regression model with a single explanatory variable:

“That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the x and y coordinates in a Cartesian coordinate system) and finds a linear function (a non-vertical straight line) that, as accurately as possible, predicts the dependent variable values as a function of the independent variables. The adjective simple refers to the fact that the outcome variable is related to a single predictor.”

OnlineStatBook (reference below) says:

“In simple linear regression, we predict scores on one variable from the scores on a second variable. The variable we are predicting is called the criterion variable and is referred to as Y. The variable we are basing our predictions on is called the predictor variable and is referred to as X. When there is only one predictor variable, the prediction method is called simple regression. In simple linear regression … the predictions of Y when plotted as a function of X form a straight line.”

“Linear regression consists of finding the best-fitting straight line through the points. The best-fitting line is called a regression line. The black diagonal line in Figure 2 is the regression line and consists of the predicted score on Y for each possible value of X. The vertical lines from the points to the regression line represent the errors of prediction. As you can see, the red point is very near the regression line; its error of prediction is small. By contrast, the yellow point is much higher than the regression line and therefore its error of prediction is large.

“The error of prediction for a point is the value of the point minus the predicted value (the value on the line). Table 2 shows the predicted values (Y’) and the errors of prediction (Y-Y’). For example, the first point has a Y of 1.00 and a predicted Y (called Y’) of 1.21. Therefore, its error of prediction is -0.21.

“… By far, the most commonly-used criterion for the best-fitting line is the line that minimizes the sum of the squared errors of prediction. That is the criterion that was used to find the line in Figure 2.”

Atlas topic, subject, and course

Regression (core topic) in Quantitative Methods and Atlas104 Quantitative Methods.

Sources

Wikipedia, Simple linear regression, at https://en.wikipedia.org/wiki/Simple_linear_regression, accessed 12 June 2017.

David Lane, OnlineStatBook, at http://onlinestatbook.com/2/regression/intro.html, accessed 12 June 2017.

Page created by: Ian Clark, last modified 12 June 2017.

Image: OnlineStatBook, at http://onlinestatbook.com/2/regression/introM.html, accessed 12 June 2017.