The standard error of estimate and its importance in regression

Concept. A regression line gives an estimated value of the dependent variable for each value of the independent variable. The actual values scatter around the line. The standard error of estimate measures that scatter: it is the standard deviation of the actual values about the regression line, an average size of the error made when the line is used to estimate. The syllabus names it as part of regression analysis. It plays for the regression line the part the standard deviation plays for the mean.

Formula. For the regression of Y on X, with Yc the value the line gives for each X,

Syx = √[Σ(Y − Yc)² / N] = σy √(1 − r²), and for X on Y, Sxy = σx √(1 − r²).

Here (1 − r²) is the coefficient of non-determination, the share of the variance the regression does not explain, so the standard error is the unexplained part of the spread, in the unit of the dependent variable.

Importance in regression analysis.

1. Reliability of estimates. A small standard error means the points lie close to the line and estimates are dependable; a large one warns that estimates are rough.
2. Measure of goodness of fit. Syx = 0 when r = ±1 (every point on the line) and Syx = σy when r = 0 (the line is no better than the mean), so it shows how far the line improves on using the average alone.
3. Range of estimates. For roughly normal scatter, about 68% of actual values lie within Yc ± 1 Syx and about 95% within Yc ± 2 Syx, so a single estimate can be given a band, for example a sales forecast of ₹82 lakh ± ₹4 lakh.
4. Comparison of lines. Between two regression lines fitted to the same dependent variable, the one with the smaller standard error is the better predictor.