Adequacy of Regression Models

Autar Kaw, Jamie Trahan
University of South Florida
United States of America
kaw@eng.usf.edu

ClearAll;

Introduction

This worksheet allows you to determine whether a straight line regression model adequately describes n data points ((Subscript[x, 1]),  (Subscript[y, 1])), ( (Subscript[x, 2]), (Subscript[y, 2])) , ((Subscript[x, 3]),  (Subscript[y, 3])),...., ( (Subscript[x, n]),  (Subscript[y, n])).  Four different checks are used to find if the model is adequate. Our discussion, although limited to straight line models, is applicable to any regression model.

Adequacy Check #1: Plot of straight-line regression model vs. data to visually inspect how well the data fits the line.
Adequacy Check #2: Calculation of the coefficient of determination, (r^2). This value quantifies the percentage of the original uncertainty in the data that is explained by the straight line model.  
Adequacy Check #3: Determine if the residuals as a function of x show nonlinearity. This is an indication that the model is not adequate.
Adequacy Check #4: Determine if 95% of the values of scaled residuals are within [-2,2]. If so, this is an indication that the model may be adequate.

Please note that the above checks are not a complete test of the adequacy of regression models. Other tests include testing if indeed y is dependent on x, confidence intervals of the constants of the model, etc.

To learn more about the quality of a fitted linear regression model, see the worksheet on the adequacy of regression models.

Section 1: Input Data

Below are the input parameters to begin the simulation. This is the only section that requires user input. 

Input Parameters:

Array of x values, X
NOTE: The user has the option of choosing his or her own X and Y array (e.g. X = {1,7,13,19,25}, Y = {1,49,169,361,625}). We are instead showing a large data set generated within a loop to better illustrate model adequacy.

X = Array[0, 50];
For[i = 1, 
      i ≤ 50, i++, 
X[[i]] = 
        N[i/2]]
X

Array of y values, Y

Y = Array[0, 50];
For[i = 1, 
      i ≤ 50, i++, 
Y[[i]] = 
        X[[i]]^2]
Y

Section 2: Finding the straight-line model

We will use Mathematica's Fit command to fit the data to a straight line.

n := Length[X]
observed = 
        Table[{X[[i]], Y[[i]]}, {i, 1, 
            n}];
f[x_] = Fit[observed, {1, x}, x];
Print["The straight-line regression model is y = ", 
      f[x]]

Section 3: Checking for the adequacy of the model

Adequacy Check #1

Below, the linear regression model is plotted versus the data points. See if the straight-line regression model visually explains the data.

points = 
        ListPlot[observed, PlotStyle -> PointSize[ .01], 
          DisplayFunction -> Identity];
lin = 
        Plot[f[x], {x, Min[X], Max[X]}, 
          DisplayFunction -> Identity];
Show[points, lin, PlotLabel -> "y vs x", 
      AxesLabel -> {x, y}, 
      DisplayFunction -> $DisplayFunction]

Adequacy Check #2

In this section we will calculate the coefficient of determination, (r^2)

(r^2)= (((Subscript[S, t])-(Subscript[S, r]))/((Subscript[S, t])))

where 
(Subscript[S, r]) = the sum of the squares of the residuals ( a value that quantifies the spread around the regression line)
and
(Subscript[S, t]) = the sum of the squares of deviation from the mean ( a value that measures the spread between the data and its mean)

This value describes the proportion of variation in the response data that is explained by the regression model. When all the points in a data set lie on the regression model, the largest possible value of (r^2)= 1 is obtained, while a minimum possible value of (r^2)=0 is obtained when there is only one data point or if the straight line regression model is a constant line. 
Note: Please see the Adequacy of Models worksheet for limitations in the use of (r^2).

Calculation of (r^2):

Sum of the difference between observed values and average values, (Subscript[S, t]):

St = (∑_(i = 1)^n)((Y[[i]] - Mean[Y])^2)

Sum of the square of the residuals, (Subscript[S, r]):

Sr = (∑_(i = 1)^n)(Y[[i]] - f[X[[i]]])^2

Coefficient of determination, (r^2):

r2 = (St - Sr)/St

Adequacy Check #3

In this section, the residuals, which are the differences between the observed values and predicted values ((Subscript[y, i]) - (Subscript[a, 0]) - (Subscript[a, 1]) (Subscript[x, i])), are found and then plotted as a function of x to check for increasing variance, outliers, or nonlinearity.

Calculating the residuals:

residuals = Array[0, {n}];
For[i = 1, i ≤ n, i++, 
        residuals[[i]] = 
          0; 
residuals[[i]] = (Y[[i]] - 
              f[X[[i]]])];
Print["The residuals are ", residuals]

Plotting the residuals:

r = 
        Table[{X[[i]], residuals[[i]]}, {i, 1, 
            n}];
residplot = 
      ListPlot[r, PlotStyle -> PointSize[0.01], 
        AxesLabel -> {x, y}, PlotLabel -> "Residuals"]

Adequacy Check #4

In this section, the scaled residuals SR (ratio between residual and the standard error of estimate) are calculated.  SR is given by

SR = (((Subscript[y, i])-f((Subscript[x, i])))/(√((Subscript[S, r])/(n - m))))

where f(x) is the regression function, n is the number of data points and m is the number of degrees of freedom lost (i.e.- the number of constants in the model. For a straight-line model, m = 2).  The procedure checks if  95% of the values are in the [-2,2] range.


Calculation of SR:

m = 2;
SR = Array[0, {n}];
For[i = 1, i ≤ n, i++, 
        SR[[i]] = 0; 
SR[[i]] = 
          residuals[[i]]/(Sqrt[Sr/(n - m)])];
count = 0;
For[i = 1, 
      i ≤ 
        n, i++, 
If[-2 <= SR[[i]] ≤ 
          2, count = count + 1]]
If[count/n >= 0.95, 
      Print[(count/
              n)*100, "% of SR values fall between -2 and 2, therefore at \
least 95% of SR values are in the [-2,2] range"], 
      Print["Only ", (count/
              n)*100, "% of SR values fall between -2 and 2, therefore at \
least 95% of the SR values do not fall in the [-2,2] range"]]

Plotting the scaled residuals:

scaleresid = 
        Table[{X[[i]], SR[[i]]}, {i, 1, 
            n}];
pointplot = 
      ListPlot[scaleresid, PlotStyle -> PointSize[0.01], 
        AxesLabel -> {x, y}, PlotLabel -> "Scaled Residuals", 
        DisplayFunction -> Identity]
lineplot = 
        Plot[{y = 2, y = -2}, {x, Min[X], Max[X]}, 
          DisplayFunction -> Identity];
Show[pointplot, lineplot, 
      DisplayFunction -> $DisplayFunction]

Conclusion

Using Mathematica we are able to check for the adequacy of a straight-line regression model. 

Question 1: Given data

({{X, 1, 5, 3, 2, 10, 20}, {Y, 1, 25, 9, 4, 100, 400}}) ,

show if regressing the data to y = (Subscript[a, 0]) +(Subscript[a, 1])x  is adequate. 

Question 2: Theoretical considerations assume that the rate of flow from a fire hose is proportional to some power of the nozzle pressure. However a scientist believes that the simpler linear regression model is adequate. Determine whether the linear model is adequate.

({{Flow rate (gallons/min)   F, 91, 120, 127, 190, 240, 310, 409}, {Pressure (psi)    p, 9, 15, 23, 40, 61, 72, 90}})

Question 3: Given ((Subscript[x, 1]),  (Subscript[y, 1])), ( (Subscript[x, 2]), (Subscript[y, 2])) , ((Subscript[x, 3]),  (Subscript[y, 3])),...., ( (Subscript[x, n]),  (Subscript[y, n])), the coefficient of determination, (r^2) is zero if the straight line regression model turns out to be a constant line. Prove that the constant line is the average of the y-values, that is \(TraditionalForm\`\(∑\_\(i = 1\)\%\(\(n\)\(\ \ \ \ \ \ \)\)y\_i\)\/n\).

References

[1] Autar Kaw, Holistic Numerical Methods Institute, http://numericalmethods.eng.usf.edu/nbm, See
On adequacy of regression models
How does Linear Regression work?