7 –4.915 24.154 47.205 –1.753 3.071 –6.968 48.559
1997 48.394 46.2009 –4.409 19.437 47.742 –1.541 2.376 –6.220 38.692
1998 52.469 50.9291 –0.334 0.111 52.068 –1.139 1.297 –1.492 2.226
1999 51.519 51.1112 –1.284 1.648 51.060 0.052 0.003 –1.310 1.716
2000 50.087 49.6888 –2.716 7.375 49.539 0.149 0.022 –2.732 7.466
2001 49.472 49.1693 –3.331 11.094 48.887 0.283 0.080 –3.252 10.575
2002 50.858 50.7086 –1.945 3.782 50.358 0.351 0.123 –1.713 2.933
2003 50.029 50.7581 –2.774 7.693 49.478 1.280 1.639 –1.663 2.766
2004 50.737 52.2648 –2.066 4.267 50.229 2.035 4.143 –0.156 0.024
2005 51.33 52.3326 –1.473 2.169 50.859 1.474 2.172 –0.089 0.008
2006 52.242 53.1504 –0.561 0.314 51.827 1.323 1.751 0.729 0.532
2007 54.266 55.5693 1.463 2.141 53.975 1.594 2.540 3.148 9.911
2008 57.019 57.6483 4.216 17.777 56.898 0.751 0.563 5.227 27.323
2009 57.564 57.3914 4.761 22.670 57.476 –0.085 0.007 4.970 24.703
2010 56.326 55.6805 3.523 12.414 56.162 –0.482 0.232 3.259 10.623
2011 57.118 56.2641 4.315 18.622 57.003 –0.739 0.546 3.843 14.768
2012 57.818 57.4775 5.015 25.153 57.746 –0.268 0.072 5.056 25.567
2013 57.02 55.9142 4.217 17.786 56.899 –0.985 0.969 3.493 12.201
2014 53.821 52.4176 1.018 1.037 53.503 –1.085 1.178 –0.004 0.000
∑Y
i = 1,056.05 ∑X i = 1,048.42 ∑ (X i − X)2 = 207.073 ∑ (Yi − Yi)2 = 24.311 ∑ (Yi − Y)2 = 257.630
Y = 52.803 X = 52.421
627
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
 ■ Standard Error of the Regression
The standard error of the regression (SER) is the standard deviation of the residuals, or equivalently, the
standard deviation of the observations from the fitted regression line:7
SER =
−()
−
=
∑ YY
n
ii
i
n
ˆ 2
1
2
This formula should look familiar. The SER was previously denoted by an s and appeared in the
calculation of the standard error of the regression coefficient, s.e.(b). The name standard error of the
regression merely highlights the fact that this figure is a measure of dispersion for the entire equation.
Note that the wider the scatter of points from the regression line, the larger the SER.
It should be emphasized that the SER can only be interpreted relative to the range of the dependent
variable. For example, in price-forecasting equations, an SER figure of 10¢ would be indicative of an
excellent fit if the given price series ranged between $6 and $12, and an extremely poor fit if the price
range were $0.30 to $0.60. For this reason, as long as the mean of the dependent variable
Y is greater
than the range between the high and low values, it may be useful to consider the percent (%SER)
where
8
%SER SER= Y
In effect, the %SER is a dispersion measure normalized by the magnitude of the underlying data.
When comparing different equations with the same dependent variable, over the same time interval, the
%SER will yield the same conclusions as the SER, with the advantage of being stated in terms that are
intuitively more meaningful.
9
 ■ Confidence Interval for an Individual Forecast
Assume that we used our regression equation to forecast a future value of the dependent variable
Y, given a specific value of the independent variable X. What forecast range would have a 95 per-
cent probability of containing the true value of Y? (Implicit assumption: The estimated value for the
7 The SER is also often referred to as the standard error of the estimate (SEE), or simply as the standard
error (SE).
8 If Y is less than the range, the %SER could be very misleading. For example, if the dependent variable included
values that were both positive and negative, its mean could be close to 0. In this case, the %SER could approach
infinity.
9 Caution should be exercised in drawing any conclusions from comparisons that involve equations with different
dependent variables, since the %SER would be sensitive to the dependent variable chosen. For an example of
why this is undesirable, see the discussion in the section “Coefficient of Determination (r
2).”
628
Appendix C
 independent variable is either known or precisely projected; that is, there is no forecast error involved
in the X value.) In answering this question, we note that even if all the regression assumptions are
fulfilled, there are three sources of potential error:
 1. error of the mean. The true population regression line is unknown and is estimated from the
observations. The resulting fitted line passes through (, )XY , while the population line passes
through (, )XY X , where in general, YX, the unknown population mean at X, will not equal the
sample mean Y . As indicated in Figure C.4a, this type of error will result in all forecasts (i.e.,
projections of Y for any value of X) being too high or low by the difference between the value
of Y and YX. (Figure C.4a depicts the case of a positive error of the mean; a symmetric image
would apply for a negative error.)
 2. error of the slope. There will also be some difference between the true regression coefficient
β and the fitted line slope b. Figure C.4b illustrates the combined effect of this source of error
and the error of the mean for the forecasted value
ˆY f at X f . Note that the error of the slope
will be zero at X since the fitted regression must pass through (, )XY , but will increase steadily
as X values are further removed from X. (The illustration in Figure C.4b depicts the case of a
positive error of the mean and an estimated regression coefficient that is too high; a symmetric
image would apply for the reverse case.)
 3. Random error. For reasons previously explained, even if the true population line were pre-
cisely known, there would still be error terms. The confidence interval for the population line
is illustrated in Figure C.4c. Note that the interval is independent of the value of X.
The forecast error for any individual prediction would reflect the combined effect of all three
of the influences just discussed (Figure C.4d). The shape of the confidence interval depicted in
Figure C.4d is a consequence of the fact tha


d still be error terms. The confidence interval for the population line
is illustrated in Figure C.4c. Note that the interval is independent of the value of X.
The forecast error for any individual prediction would reflect the combined effect of all three
of the influences just discussed (Figure C.4d). The shape of the confidence interval depicted in
Figure C.4d is a consequence of the fact that, although the error of the mean and the random error
components will be equal for all values of X, the error of the slope increases for values further
removed from
X. If the independent variable value for the forecast period is denoted by Xf, the stan-
dard error for the forecasted value of Yf is given by
se.. ()ˆYs n
XX
XX
f
f
i
i
n=+ +
−()
−()
=
∑
1 1
2
2
1
Note that when Xf, equals X, and n is large, this term reduces to approximately s—the standard
deviation of the residuals. (Once again, s is more commonly called the standard error of the regres-
sion.) Also note that the further X
f is from X, the larger the s.e. ()Y fˆ
The confidence interval for Yf would be
ˆ .. ( ˆ ) ˆ .. ( ˆ )Yt YY Yt Yff ff f−< <+ ⋅∑ se se
629
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
Fitted regression line
Error of the mean
Unknown population line
Y
(a)
XX
X
Y
YX
Fitted regression line
Error of the mean
Line parallel to unknown population line
Unknown
population
line
Y
(b)
XX
Y
Yf
Xf
95% confidence interval
if population line known
Population line
Y
(c)
XXf
95% confidence interval for
individual forecast at X = Xf
Y
(d)
XXf
Error of the
slope
Fitted regression line
95% confidence interval
for individual forecast at
mean of X
 FIGURE  C.4 Confi dence Interval for an Individual Forecast
where t = t value at the specifi ed level of signifi cance for the given number of degrees of freedom. At
XX= , this interval would reduce to
ˆˆYt s n YY ts nff f−⋅ +< <+ ⋅+1 1 1 1
630
Appendix C
 ■ Extrapolation
Extrapolation refers to predictions beyond the range of observations, that is, forecasts for Yf where Xf
is greater than or less than any observed value Xi. As should be apparent from the preceding formula
for s.e. (Yf), predictions in the extrapolated range will be particularly subject to uncertainty, since s.e.
( ˆY f ) increases as the X value moves farther away from X. However, there is an even more important
reason why extrapolation-based forecasts should be viewed with great skepticism. Basically, it is never
safe to assume that the relationship between the dependent and independent variables exhibited in the
observed range would continue to hold in the extrapolated range. For example, consider a market in
which there is a rough inverse linear relationship between price and the final stock/consumption ratio
during the observation period. This relationship would likely break down in a record shortage situa-
tion, since prices frequently begin to rise at an accelerated rate once an expected shortage reaches a
critically low level.
What should be done in the case in which the expected value for the explanatory variable falls
beyond the observed range? Many professional analysts faced with such a dilemma will apprehen-
sively extrapolate and hope for the best (probably because of implicit pressure to provide forecasts,
whether the necessary data exists or not). Such hopes are often misplaced. This does not mean to
imply that the analyst must resign herself to a yearlong hibernation before she ventures any market
forecast—few firms are that enlightened. Rather, the intended point is that in such a situation, the
analyst must rely almost totally on her intuitive fundamental sense of the market and technical
analysis to generate forecasts, rather than naively continuing to use a formal fundamental model that
is no longer relevant.
 ■ Coefficient of Determination (r2)
If the regression model were unavailable or, equivalently, if the independent variable were useless in
explaining the dependent variable Y, the best forecast for a value of Y would be its mean
Y . W e define
the difference between an individual observation and the mean, YYi − , as the total deviation. Now
if X is of any use in explaining Y , the deviations between the observed points and the fitted regres-
sion line should tend to be smaller than the total deviation. For any given observation Yi, the portion
of the total deviation explained by the regression equation would equal the fitted value minus the
mean,
ˆYYi − . The portion that remains unexplained will equal the observed value minus the fitted
value YYii − ˆ . The relationship among the explained, unexplained, and total deviations is illustrated in
Figure C.5. For any given observation, this relationship can be expressed as follows:
Total deviatio n= explained deviatio n+ unexplai ned devi ation
foor forf orYY Y
YY YY YY
ii i
ii ii−() =− () −()+ˆˆ
If we are interested in deriving a relationship between total variation, a measure of the deviation of
all points, and explained and unexplained variation, we cannot simply sum the terms, because opposite
631
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
deviations will off set each other, yielding a tautological relationship. 10 Thus, we square both sides of
the equation before summing. This step is analogous to the approach used to get around the same
problem in trying to fi nd the best-fi t line:

YY YY YY
YY YY
i
i
n
ii i
i
n
ii i
−() =− () +−() 

=−
() +− ()
==
∑∑
2
1
2
1
22
ˆˆ
ˆˆ −−− () −()
===
∑∑∑ 2
111
YY YYii i
i
n
i
n
i
n
ˆˆ

 Given the previously stated assumption that the independent variable X and the error terms are
uncorrelated, it can be algebraically demonstrated that
20
1
YY YYii i
i
n
−() −() =
=
∑
 10 YY YY YY
YY Y
i
i
n
i
i
n
ii
i
n
i
i
n
i
i
n
−() =− () +− ()
=− +
== =
==
∑∑ ∑
∑∑
11 1
11
ˆˆ
ˆ

− −
=− () =−
=
==
==
∑∑
∑∑
ˆY
YY Yn Y
i
i
n
i
n
i
i
n
i
i
n
11
11
00
 FIGURE  C.5 Explained, Unexplained, and T otal Deviation
Total deviation (Yi − Y)
Y = a + bXY
X
Explained deviation (Yi

eBooks\A Complete Guide to the Futures Market Technical Analysis, Trading Systems, Fundamental Analysis, Options, Spreads\A Complete Guide to the Futures Market Technical Analysis, Trading Systems, Fundamental Analysis, Options, Spreads.pdf


he error terms are
uncorrelated, it can be algebraically demonstrated that
20
1
YY YYii i
i
n
−() −() =
=
∑
 10 YY YY YY
YY Y
i
i
n
i
i
n
ii
i
n
i
i
n
i
i
n
−() =− () +− ()
=− +
== =
==
∑∑ ∑
∑∑
11 1
11
ˆˆ
ˆ

− −
=− () =−
=
==
==
∑∑
∑∑
ˆY
YY Yn Y
i
i
n
i
n
i
i
n
i
i
n
11
11
00
 FIGURE  C.5 Explained, Unexplained, and T otal Deviation
Total deviation (Yi − Y)
Y = a + bXY
X
Explained deviation (Yi − Y)
Unexplained deviation
(Yi − Yi)
Y = Y
632
Appendix C
Thus, we have
Total variatio ne xplai ned vari ation unexplai ned vari ation=+
Yii i
i
n
ii
i
n
i
n
YY YY Y−() =− () +− ()
===
∑∑∑
2 2
1
2
11
ˆˆ
where variation is defined as the sum of the squared deviations.
Dividing both sides by YYi
i
n
−()
=
∑
2
1
1
1
2
1
2
1
2
1
2
1
=
−()
−()
+
−()
−()
=
=
=
=
=
∑
∑
∑
∑
ˆˆYY
YY
YY
YY
i
i
n
i
i
n
ii
i
n
i
i
n
exp plained vari ation
total variatio n
unexplai ned vari ation
tot+ aal vari ation
W e define r2 as the
Explai ned vari ation
Total variatio n =
−()
−()
=
=
∑ ˆYY
YY
i
i
n
i
i
2
1
2
1
n n
∑
or equivalently,
r
YYii
i
n
2
2
111=− =−
−()
=
∑unexplai ned vari ation
total variatio n
ˆ
YYYi
i
n
−()
=
∑
2
1
The second form is more convenient because the analysis of the regression equation focuses on the
unexplained variation (residuals). Note that 0 ≤ r2 ≤ 1. If X does not explain any of the variation in Y,
then r2 = 0. If X explains all of the variation in Y (i.e., all the observations fall precisely on the fitted
line), r2 = 1. The r2 calculation is an extremely useful summary statistic because of the significance of
what it measures—the percentage of variation explained by the regression equation—and the intui-
tive clarity with which the r2 figure can be interpreted. Table C.1b illustrates the calculation of r2 for
the regression line derived in Table A.1.
The statistic r 2 is extremely useful in comparing alternative models, as long as they all have
the same dependent variable, that is, they differ only in the explanatory variables. However, when
633
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
 regression equations have different dependent variables, inferences based on r 2 can prove very mis-
leading. For example, consider the following two models:
Model I:
Model II:
Pa bP r
Pa bX r
tt
t
=+ =
=+ =
−1
2
2
09 8
05 0
,.
,.∆
where Pt = closing price on day t
Pt−1 = closing price on day t − 1
ΔPt = Pt − Pt−1
X = an explanatory variable known on day t − 1 and used to predict price changes
Despite the fact that Model I has a much higher r2, Model II represents the better forecasting equation.
If the survey period is sufficiently long, an equation such as Model I will merely tell us that the price on a
given day will be almost equal to the preceding day’s price. (In such an equation, b is likely to be very close
to 1.0.) The reason for the high r2 value is that although prices for the entire period may range widely,
prices on adjacent days must be closely correlated (at most, they can be separated by the daily price
limit). Model I, however, is totally useless in forecasting the next day’s price. In contrast, the independent
variable in Model II explains 50 percent of the price change on a given day and might be an extremely
important aid in day trading. This illustration is intended to highlight the potential folly of making value
judgments about regression equations with different dependent variables.
11
11 An even more extreme example is possible. If we used hogs not slaughtered (HNS) instead of hog slaughter
(HS) as the dependent variable regressed against the pig crop (PC), where HNS = PC − HS, the sum of the
squared residuals and hence SER would be exactly the same, but r 2 would be different. Why? Because r2 would
be affected by the dependent variable chosen
r
YYii
i
n
2
2
111=− =−
−()
=
∑unexplained variatio n
total variatio n
ˆ
YYYi
i
n
−()
=
∑
2
1
In this example,
YYii
i
n
−()
=
∑ ˆ 2
1
will be the same whether HS or HNS is the dependent variable, but
YYi
i
n
−()
=
∑
2
1
will be different, hence r2 will be different.
634APPENDIX C
 ■ Spurious (“Nonsense”) Correlations
 It is important to understand that cause and eff ect in the regression procedure are in the eyes of the
beholder. The r 2 value derived in Table C.1 b merely tells us that there is a strong correlation between
hog slaughter and the preceding pig crop. The way we interpret the cause and eff ect of this statistic
emanates only from our theoretical understanding of the underlying process. In this particular exam-
ple, it is quite obvious the pig crop in one period aff ects slaughter in the next period rather than vice
versa. However, if enshrouded in ignorance, we set out to prove the level of hog slaughter determines
the number of pigs born in the preceding period, the resulting equation would yield the identical r
2
value. Thus, r 2 only refl ects the degree of correlation between two variables; it in no way proves a
cause-and-eff ect relationship.
 The potential folly of drawing cause-and-eff ect inferences from an r 2 fi gure is demonstrated by
Figure C.6 . Note what appears to be a striking relationship between the number of hedge funds
and U.S. wine consumption. In fact, the r
2 value between the number of hedge funds and U.S. wine
consumption during the period depicted is a remarkably high 0.99! What conclusions are we to draw
from this chart?
 ■ Increased wine consumption encourages people to invest in hedge funds.
 ■ Hedge funds drive people to drink.
 ■ The hedge fund industry should promote wine consumption.
4,000
4,500
5,000
5,500
6,000
6,500
7,000
7,500
8,000
8,500
9,000
650600550500450
Number of Hedge Funds
Wine Consumption (Mil. Gal.)
95
96
97
98
99
00
0201
03
04
r2 0.99
 FIGURE  C.6 Number of Hedge Funds vs. U.S. Wine Consumption
635
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
 ■ Wine growers should promote hedge fund investing.
 ■ All of the above.
 ■ None of the above.
Actually, the striking correlation between wine consumption and the number of hedge funds i


00450
Number of Hedge Funds
Wine Consumption (Mil. Gal.)
95
96
97
98
99
00
0201
03
04
r2 0.99
 FIGURE  C.6 Number of Hedge Funds vs. U.S. Wine Consumption
635
CHECKING THE SIGNIFICANCE OF THE REGRESSION EQUATION
 ■ Wine growers should promote hedge fund investing.
 ■ All of the above.
 ■ None of the above.
Actually, the striking correlation between wine consumption and the number of hedge funds is
very easily explained. Both variables were affected by a common third variable during the period
depicted: time. In other words, both the number of hedge funds and wine consumption witnessed
pronounced growth trends during this time period. The apparent relationship arises from the fact
that these trends were simultaneous. This type of coincident linear relationship is called “spurious” or
“nonsense” correlation. Actually, the correlation is real enough; only the interpretation of cause and
effect is nonsense.

637
The Multiple
Regression Model
Appendix d
In our description of nature the purpose is not to disclose the real essence of the
phenomena but only to track down, so far as it is possible, relations between
the manifold aspects of our experience.
—Niels Bohr
 ■ Basics of Multiple Regression
In practice, it is rarely possible to explain the behavior of a dependent variable adequately with only
one explanatory variable. For example, hog slaughter alone will provide only a rough indication of hog
prices. A more satisfactory model would also incorporate other independent variables, such as broiler
slaughter. The multiple regression equation is a straightforward extension of simple regression and describes
the linear relationship between the dependent variable and two or more independent variables.
The meaning of linear, which might not be intuitively obvious beyond the two-dimensional case,
is that all the variables are of the first degree and are combined only by addition or subtraction. For
example, in terms of Z as a function of X and Y, Z = 2X + Y + 3 is a linear equation, while Z = X
2 + 2y2
+ 4, Z = XY, and Z = log X + log Y are nonlinear equations. A basic characteristic of a linear equation
is that a one-unit change in an independent variable will result in a constant magnitude change in the
dependent variable, regardless of the independent variable value. In other words, in a linear equation,
the slope in each dimension is constant. When there are only two variables, as is the case in simple
regression, the linear equation can be depicted by a straight line. When there are three variables, the
linear equation can be represented by a plane in three-dimensional space. Linear equations involving
more than three variables can no longer be simply represented in three-dimensional Euclidean space.
638
Appendix d
As in the simple regression case, regression analysis is only appropriate if the relationship between
the variables is approximately linear. This is not as strict a limitation as it may sound, since many non-
linear equations can be transformed into linear equations.
The general form of the multiple regression equation is
YX XX ekk=+ +… +αβ ββ11 22
The preceding equation represents the unknown population or true regression. The general form
of the fitted regression is
Ya bX bX bXkk=+ +…11 22
where a, b1, b2, . . . , bk are chosen so as to minimize the sum of the squared residuals. A regression
coefficient bi can be interpreted as follows: If all other independent variables are held constant, a
one-unit change in X i will cause the dependent variable Y to change by b i. A residual can still be
interpreted as the difference between an observed value of the dependent variable (Yi) and the fitted
value (Yi). Regardless of how many variables there are in the equation, the residuals still represent a
single-dimensional difference.
These concepts can perhaps be clarified by considering a practical example. Assume that we have
derived a regression equation relating hog prices to hog slaughter and broiler slaughter:
Ya bX bX=+ +11 12 2
where Y = deflated hog prices
X1 = hog slaughter
X2 = broiler slaughter
This relationship is depicted in Figure D.1. Each combination of values for X 1 and X 2 will fix a
location in the (X1, X2) plane. The regression equation will indicate the value of Y (the height of the y
axis) at that point. In other words, the regression equation defines the price level that corresponds to
any combination of hog and broiler slaughter.
For any given value of X
2 (broiler slaughter), increases in hog slaughter will reduce Y (hog prices)
at a constant rate. Similarly, for any given value of X 1 (hog slaughter), increases in broiler slaughter
will reduce Y. Thus, as can be seen, the highest prices will occur when hog slaughter and broiler
slaughter are low , and the lowest prices when these explanatory variables are high.
Each observation will represent a combination of values for X1 and X2 and the corresponding value
of Y during the given period. Of course, the actual observed Y i values, which are indicated by solid
dots in Figure D.1, will rarely fall exactly on the plane. The vertical distance between any point and
the plane (i.e., the difference between the actual hog price and the price predicted by the regression
plane) is the residual and is indicated by an arrow . As in the simple regression case, the residual is
measured along a single axis (y). The regression procedure will specify the values for a, b
1, and b2,
which will minimize the sum of the squares of these residuals.
639
THE MULTIPLE REGRESSION MODEL
 W e deliberately avoid indicating the computational formulas for deriving the regression coef-
fi cients, or for that matter, any of the statistics for the multiple regression case. The reason for this is
that multiple regression computations are far too cumbersome to be reasonably considered without
the aid of a computer. The assumptions underlying multiple regression are completely analogous to
those detailed fo


id indicating the computational formulas for deriving the regression coef-
fi cients, or for that matter, any of the statistics for the multiple regression case. The reason for this is
that multiple regression computations are far too cumbersome to be reasonably considered without
the aid of a computer. The assumptions underlying multiple regression are completely analogous to
those detailed for the simple regression model. In the multiple regression case, there is one additional
assumption: that there is no linear relationship among any two or more independent variables. If this
assumption is not met, it will lead to a problem called multicollinearity .
 FIGURE  D.1 Scatter of Observed Points About the Regression Plane
X1 = hog slaughter
Y = a + b1X1 + b2X2
X2 = broiler slaughter
Y = deflated hog price
640
Appendix d
 ■ Applying the t-Test in the Multiple Regression Model
The t values in the multiple regression output can be used to evaluate the significance of the regres-
sion coefficients. The t values from any standard computer regression analysis assume the hypothesis
being tested is that the regression coefficient equals zero. In this case, the t value for bi is provided by
t b
b
i
i
= se.. ()
This relationship can be expressed as follows:
T-STA T COEFF
ST ER=
where T -STAT = t value for given regression coefficient
COEFF1 = value of given regression coefficient
ST ER = standard error of given regression coefficient (not to be confused with the standard
error of the regression, SER, which is described in the next section)
Frequently, these statistics will be listed in three adjacent columns with each row providing sta-
tistics for the constant term or specified regression coefficient. The interpretation of the t statistic
is identical to the simple regression case. The t value indicates the number of standard deviations
between the indicated coefficient value and the true population coefficient if the latter were actually
equal to zero. The higher the t value, the more significant the regression coefficient. T o use the t table
(Table B.4), one would check the row corresponding to n−k degrees of freedom (df ) where n equals
the total number of observations and k equals the number of variables in the equation. Roughly speak-
ing, t values above 2.0 are clearly significant and indicate that the given independent variable should
be retained in the model. It should be noted that the t value does not definitively prove significance;
rather it establishes significance at a specific probability level. The higher the t value, the more unlikely
a regression coefficient would be assumed to be significant when it is not.
What if the t statistic is less than the t value at the 0.05 level of significance (e.g., less than 1.812
for a one-tailed test with 10 df)?
2 Here there is no clear-cut answer. The choice depends on the priori-
ties of the analyst. If she is more concerned about retaining an insignificant variable in the model, she
might lean to dropping any variable whose regression coefficient is not significant at the 0.05 level.
On the other hand, if she is more concerned about deleting a meaningful variable, she would retain
the variable unless the t value was very low .
A reasonable criterion is that any theoretically meaningful variable with a t value greater than
1.0 should usually be retained,
3 although it should be noted that many analysts prefer to use a cutoff
1 Frequently also called VALUE.
2 It is assumed that the regression coefficient has the anticipated sign (e.g., negative for hog slaughter in an equa-
tion in which hog prices are the dependent variable). Situations in which the sign is opposite to expectations are
discussed later in this appendix.
3 There is a special meaning to t values > 1.0. It has been demonstrated that if explanatory variables are retained
if their t-value > 1.0 and deleted otherwise, the “Corrected R2” (which is discussed later) will be maximized.
641
THE MULTIPLE REGRESSION MODEL
level of 2.0. The key words are theoretically meaningful. A low t value does not contradict the assumed
relationship between the dependent and explanatory variable. Remember, a t value below the level of
statistical significance does not indicate that the independent variable is not meaningful in explaining
the dependent variable. It only means that its significance has not been demonstrated at the desired
probability level. As long as the variable has the anticipated sign, the results are still consistent with
theoretical expectations, albeit the relationship is not as strong as would be desired. Furthermore,
even a t value of 1.0 would still be significant at the 0.20 level (i.e., 80 percent probability) for any
regression equation in which df
 > 2. Variables with t values below 1.0 should usually be dropped.
There is one exception to the decision process just detailed. Occasionally, the analyst might try
including all the independent variables she believes should significantly affect the dependent variable,
only to find that the resulting regression equation is disappointing. At this point, in desperation she
might try a variety of independent variables in the hopes that perhaps one or more of these are signifi-
cantly related to the dependent variable. Such a method could be termed a “shotgun” or “kitchen sink”
approach and is not recommended unless all theoretically plausible variables have been exhausted. In
any event, in this case one should apply stricter requirements for retaining a variable. First, a two-
tailed rather than one-tailed test should be used (see section “T esting the Significance of the Regres-
sion Coefficients” in Appendix C). Second, variables with t values below the 0.05 level of significance
should be rejected. In fact, one can argue that a more restrictive significance level should be adopted
(e.g., 0.01), since the probability of accepting a meaningless variable increases with the number of
variables t


d test should be used (see section “T esting the Significance of the Regres-
sion Coefficients” in Appendix C). Second, variables with t values below the 0.05 level of significance
should be rejected. In fact, one can argue that a more restrictive significance level should be adopted
(e.g., 0.01), since the probability of accepting a meaningless variable increases with the number of
variables tested.
Thus far, we have assumed that a theoretically chosen variable has the correct sign. However, in
equations with many variables, a coefficient with the wrong sign is not uncommon. Such an occur-
rence usually indicates the presence of multicollinearity—a linear dependence between two or more
explanatory variables. (A discussion of how to handle such variables is presented in the section on
multicollinearity in Appendix E.) At this point, suffice it to say that the t values of such variables are
usually irrelevant.
 ■ Standard Error of the Regression
The standard error of the regression (SER) is a measure of the unexplained variation. The definition of
the SER is almost totally analogous to the simple regression case. The only difference is that the sum
of the squared residuals is divided by the appropriate degrees of freedom, instead of n−2. Thus, for
the more general multiple regression case, the SER could be expressed as
SER =
−()
−
=
∑ YY
nk
ii
i
n
ˆ 2
1
where k = number of parameters in equation (which is equal to the number of independent vari-
ables plus 1, assuming there is a constant term in the equation). Note in the simple regression case
that k = 2.
642
Appendix d
As in the simple regression case, the % SER is equal to the SER divided by Y . Where appropriate
(see Appendix C), the % SER may be more convenient to use, because it is stated in a form that is
intuitively meaningful.
 ■ Confidence Intervals for an Individual Forecast
In the multiple regression case, the calculation of a confidence interval for an individual forecast is
somewhat complicated. As a simplification, the confidence interval can be calculated for the situation
in which all of the independent variables are equal to their means. In this specialized case, the formula
for the confidence interval would be analogous to the simple regression case in which
XX= :
ˆˆYt s n YY t nff f−⋅ +< <+ ⋅+1 1 1 1
where s = SER
t = t value at specified level of significance for the given degrees of freedom
This represents a minimum confidence interval, and the further removed the independent vari-
ables are from their respective means, the wider the actual confidence interval.
 ■ R2 and Corrected R2
The term R2 is the multiple regression counterpart of r2 and is defined in exactly the same way. Thus,
the entire discussion related to r2 in Appendix C applies here as well and need not be duplicated.
In the multiple regression case, it is important to realize that the addition of another independent
variable can only increase R2. Remember that R2 is the ratio of explained variation to total variation.
The introduction of a new variable will not affect total variation, and it can only increase explained
variation. Even the introduction of a totally irrelevant variable will probably result in a small increase
in explained variation. For example, it is a safe bet that adding a variable for the number of ducks in
Belgium would increase the R
2 of a regression equation for forecasting U.S. interest rates.
The point that the addition of a meaningless explanatory variable will raise R 2 is more than an
esthetic consideration. Recall that each additional variable will decrease the degrees of freedom by
1, thereby reducing the significance of the equation on the basis of other measures such as the t-test
and SER, all else being equal. For this reason, it is desirable to modify the R
2 measure so that it is
penalized for the addition of irrelevant variables. This alternative measure is called the Corrected R 2
(CR2), or sometimes the Adjusted R2. The problem with R2 is that it is based on variation, which does
not account for the number of degrees of freedom. The CR2 avoids this defect, because it is based on
variance. The variance is simply the variation divided by the number of degrees of freedom. It will be
recalled that R
2 can be defined as4
R2 1=− unexplai ned vari ation
total variatio n
4 The formulas for R2 and r2 are identical.
643
THE MULTIPLE REGRESSION MODEL
W e now define CR2 as
CR2 1=− unexplai ned vari ance
total variance
where
Variance variatio n= df
Thus,
CR
YY
nk
YY
n
i
i
n
i
n
2
1
2
1
1
2
1
1
1
=−
−
−
−
−
=
=
∑
∑
()
()

The numerator of the ratio term is based on n observations, but there are k constraints in find-
ing the regression line used to calculate Y i. Thus df = n − k. The denominator is also based on n
observations, but there is only one constraint, Y ; thus df = n − 1. The preceding equation can be
rewritten as
CR R n
nk
22 11 1=− −⋅ −
−()
As is readily apparent in this form of the equation, when n is large relative to k, the CR2 will almost
equal R2.
Typical regression runs will provide the CR2 (corrected R square or adjusted R square) along with
R2 (R square). As a general rule, the CR2 is a more useful measure for comparing different regression
equations for the same dependent variable.
 ■ F-Test
Whereas the t distribution is used to test the significance of the individual regression coefficients, the
F distribution is used to test the significance of the regression equation as a whole. In other words,
the F statistic tests the hypothesis that none of the regression coefficients is significant. The F statistic
can be expressed as
F = explained variance
unexplai ned vari ance
644
Appendix d
Note that the F value is based on variance, not variation. Once again, variance = variation + df.
F
YY
k
YY
nk
YY
YY
i
i
n
ii
i
n
i
i
n
ii
i
=
−()
−
−()
−
=
−()
−()
=
=
=
∑
∑
∑
2
1
2
1
2
1
2
1
ˆ
ˆ
ˆ
= =
∑
⋅ −
−
1
1n
nk
k
The degrees of freedom for the explained variance = k − 1, since k v


significant. The F statistic
can be expressed as
F = explained variance
unexplai ned vari ance
644
Appendix d
Note that the F value is based on variance, not variation. Once again, variance = variation + df.
F
YY
k
YY
nk
YY
YY
i
i
n
ii
i
n
i
i
n
ii
i
=
−()
−
−()
−
=
−()
−()
=
=
=
∑
∑
∑
2
1
2
1
2
1
2
1
ˆ
ˆ
ˆ
= =
∑
⋅ −
−
1
1n
nk
k
The degrees of freedom for the explained variance = k − 1, since k values are employed in
defining the regression line used to calculate ˆYi, but one df is lost because of the constraint imposed
by Y . As for the unexplained variance, there are n observations, but k constraints are imposed in
finding the regression line upon which Y i is based. Recalling the alternative definitions for R 2, we
can re-express F as5
F R
R
nk
k= − ⋅ −
−
2
211
The appropriate degrees of freedom will be specified in the notation for the F statistic. For
example, F (2/8) = 23.5 indicates an F value for a regression equation in which k − 1 = 2 and
n − k = 8. T o check for significance, the F statistic is compared to the listed values in the F table
for the corresponding number of degrees of freedom. For example, checking Table D.1, it can
be determined that at the 0.01 level of significance, F (2/8) = 8.65; thus, a value of 23.5 would
be significant.
In practice, the F-test is not particularly critical, since it will almost invariably prove significant.
This should not be surprising, because the F-test checks whether all the regression coefficients com-
bined have any predictive value—a very weak criterion. In any case, for comparisons of regression
equations with the same dependent variable, higher F values would indicate a better model (assuming
none of the regression assumptions are violated). However, similar information could be gathered by
comparing CR
2 values.
 ■ Analyzing a Regression Run
Table D.2 presents the results for a sample regression run. At this juncture, most of Table D.2 should
be comprehensible. However, it may be helpful to interpret the key statistics of this table.
 1. The regression equation is Y = 49.06899 − 1.07049 (X1) + 0.35775 (X2). T o get a point fore-
cast for Y, one would merely plug in the estimated values of X1 and X2. For example, if X1 = 20
5
ˆYY RY Yii
i
n
i
n
−() =⋅ −()
==
∑∑
2 2 2
11
 and YY RY Yii i
i
n
i
n
−() =− −
==
∑∑ ˆ () ()
2 22
11
1
645
THE MULTIPLE REGRESSION MODEL
TAble d.1 F distribution
Values of Fn1,n2,α on the F(n1,n2,α)-distribution
pr{F(n1,n2)-variable ≥ Fn1,n2,α} = α = 0.01
0
/uni03B1 = .01
/uni03B1 = .01
Fn1, n2, /uni03B1
F(n1, n2)-distribution
n1 (numerator df )
n2 (denominator df ) 1 2 4 6 8 10 12 24 ∞
[tn2,.005]2 Values of Fn1,n2,α
1 4,052 5,000 5,625 5,859 5,982 6,056 6,106 6,235 6,366
2 98.50 99.00 99.25 99.33 99.37 99.40 99.42 99.46 99.50
3 34.12 30.82 28.71 27.91 27.49 27.23 27.05 26.60 26.13
4 21.20 18.00 15.98 15.21 14.80 14.55 14.37 13.93 13.46
5 16.26 13.27 11.39 10.67 10.29 10.05 9.89 9.47 9.02
6 13.75 10.92 9.15 8.47 8.10 7.87 7.72 7.31 6.88
7 12.25 9.55 7.85 7.19 6.84 6.62 6.47 6.07 5.65
8 11.26 8.65 7.01 6.37 6.03 5.81 5.67 5.28 4.86
9 10.56 8.02 6.42 5.80 5.47 5.26 5.11 4.73 4.31
10 10.04 7.56 5.99 5.39 5.06 4.85 4.71 4.33 3.91
11 9.65 7.21 5.67 5.07 4.74 4.54 4.40 4.02 3.60
12 9.33 6.93 5.41 4.82 4.50 4.30 4.16 3.78 3.36
13 9.07 6.70 5.21 4.62 4.30 4.10 3.96 3.59 3.17
14 8.86 6.51 5.04 4.46 4.14 3.94 3.80 3.43 3.00
15 8.68 6.36 4.89 4.32 4.00 3.80 3.67 3.29 2.87
20 8.10 5.85 4.43 3.87 3.56 3.37 3.23 2.86 2.42
25 7.77 5.57 4.18 3.63 3.32 3.13 2.99 2.62 2.17
30 7.56 5.39 4.02 3.47 3.17 2.98 2.84 2.47 2.01
40 7.31 5.18 3.83 3.29 2.99 2.80 2.66 2.29 1.80
60 7.08 4.98 3.65 3.12 2.82 2.63 2.50 2.12 1.60
120 6.85 4.79 3.48 2.96 2.66 2.47 2.34 1.95 1.38
∞ 6.63 4.61 3.32 2.80 2.51 2.32 2.18 1.79 1.00
Source: Abridged from Table 18 of Pearson and Hartley, Biometrika T ables for Statisticians, Third Edition, V olume 1, 1976, with kind permission of
the Biometrika Trustees (http://biomet.oxfordjournals.org).
The diagram, and this presentation of the table, are taken from Table 4b of S. R. Searle, Linear Models (New Y ork, NY: John Wiley & Sons,
1997). Copyright © 1997 by John Wiley & Sons; reprinted by permission.
646
Appendix d
TAble d.2 Sample Regression Run Results
Variable Coefficient Standard error T-Stat Mean
CONSTANT 49.06899 9.67267 5.07 41.16071 (dependent variable)
X1 −1.07049 0.23464 −4.56 21.00714
X2 0.35775 0.13400 2.67 40.75357
Observation Actual Fitted Residual % deviation
1 35.90000 35.25925 0.64075 1.82
2 52.70000 54.54910 −1.84910 −3.39
3 46.30000 50.74680 −4.44680 −8.76
4 34.20000 36.98609 −2.78609 −7.53
5 51.30000 46.15574 5.14426 11.15
6 44.20000 44.02220 0.17780 0.40
7 33.90000 29.70675 4.19325 14.12
8 31.30000 30.54304 0.75696 2.48
9 31.70000 32.74207 −1.04207 −3.18
10 29.90000 31.58795 −1.68795 −5.34
11 51.10000 49.76981 1.33019 2.67
12 56.10000 51.62468 4.47532 8.67
13 43.90000 45.56465 −1.66465 −3.65
14 33.7500 36.99187 −3.24187 −8.76
Y = CONSTANT + C1 · X1 + C2 · X2
RSQ = 0.8953 SER = 3.2338 F(2,11) = 47.0
RSQC = 0.8762 % SER = 7.86 DW = 1.69
and X2 = 40, the predicted Y value would be 41.969. In practice, it will be more convenient to
use mnemonic symbols for the variables instead of Y, X1, and X2.
 2. R2 = 0.8953, which means that X1 and X2 explain 89.53 percent of the total variation in Y. The
CR2, which is adjusted downward for lost degrees of freedom, is 0.8762.
 3. SER = 3.2338. This would be a key figure of merit in comparisons with alternative models. The
SER could also be used to construct a crude confidence interval for an individual forecast based
on the assumption that all the independent variable values are equal to their respective means.
This confidence interval would be
6
ˆˆYt s n YY ts nff f−⋅ +< <+ ⋅+1 1 1 1
6 See the section, “Confidence Intervals for an Individual Forecast.”
647
THE MULTIPLE REGRESSION MODEL
where s = SER = 3.23
n = 14
t = 2.201 (t value for two-sided test at 0.05 level of significance for 11 df )
ˆ .( .) (. ) ˆ .( .) .)
ˆ .
(YY Y
Y
ff


all the independent variable values are equal to their respective means.
This confidence interval would be
6
ˆˆYt s n YY ts nff f−⋅ +< <+ ⋅+1 1 1 1
6 See the section, “Confidence Intervals for an Individual Forecast.”
647
THE MULTIPLE REGRESSION MODEL
where s = SER = 3.23
n = 14
t = 2.201 (t value for two-sided test at 0.05 level of significance for 11 df )
ˆ .( .) (. ) ˆ .( .) .)
ˆ .
(YY Y
Y
ff f
f
−< <
−
22 01 32 31 03512 2013 23 10 351
73 5
+
888 7 3588<<YYff ˆ . +
Using the point estimate for Y f of 41.969 derived in 1, the 95 percent confidence interval
would be
34 610 49 328.. <<Y f
This would mean a 95 percent probability the actual value will fall in the stated range if the
forecast is based on independent variables equal to their respective means. Of course, this will
never be the case. Consequently, the actual confidence interval will always be wider. Neverthe-
less, given this understanding, the simplified confidence interval provides at least a rough sense
of the potential variability of the forecast.
 4. The %SER SER=÷ Y . The %SER provides a figure that is intuitively meaningful and can be
used instead of the SER if all model comparisons involve the same dependent variable.
 5. The F value for 2 df in the numerator and 11 df in the denominator = 47.0, which is well above
the listed F value of 7.21 at the 0.01 significance level. Again, the F test will almost invariably
verify that the equation is significant.
 6. DW stands for Durbin-Watson, a measure discussed in Appendix E..
 7. The t statistic is equal to the coefficient values divided by the respective standard error. In this
example, all the coefficients are significant (t value at the 0.05 level of significance for a one-
tailed test with 11 df is 1.796).
 8. The Actual column in Table D.2 lists the actual observations of Y, and the Fitted (also called
Predicted) column lists the corresponding values indicated by the regression equation. The dif-
ference between these two figures for each observation is listed in the Residual column. The
Percent Deviation column is equal to the residual divided by the fitted value. (In some cases, the
residuals are normalized by dividing the residuals by the SER—that is, expressing residuals in
standard deviation units. Residuals normalized in this manner are termed standardized residuals
or standard residuals and are discussed in Appendix E.)
Thus far, we have only discussed the meaning of the overall summary statistics for the regression
equation. As will be detailed in Appendix E, the individual residual values also contain extremely
important information and should be carefully analyzed.

649
Analyzing the
Regression Equation
Appendix e
It is a test of true theories not only to account for but to predict phenomena.
—William Whewell
 ■ Outliers
An outlier is an observation with a large residual—that is, a large deviation between the observed
value and fitted value. Outliers reflect one of the following conditions:
 1. An error in collecting or manipulating the data for the given point.
 2. The existence of a significant extraneous causal factor that only affected the outlier(s).
 3. The omission of an important explanatory variable from the equation.
 4. A structural flaw in the model.
The presence of outliers indicates a deficiency in the model. After verifying that an outlier is not
the result of error, one should try to identify possible factors responsible for the aberrant behavior.
If the outlier can be explained by a missing variable that affected all observations, then this variable
should be included in the equation. If, however, the outlier was a consequence of an isolated event that
is not expected to reoccur, then it should be viewed as an unrepresentative point, and the regression
should be rerun with the outlier deleted. This recalculation is important, since the method of least
squares used to derive the regression coefficients will give greater weight to outliers. Thus, one or
two such points could seriously distort the regression equation fit. However, unless the isolated causes
of the outlier have been identified, one should avoid the temptation of deleting such points simply
because it will improve the regression fit.
650
Appendix e
 ■ The Residual Plot
The scatter diagram, such as the one depicted in Figure A.4, is of limited use in detecting outliers,
since it can only be applied in the simple regression case. The residual plot provides a graphic tech-
nique for detecting outliers that is as easy to apply in multiple regression as it is in simple regression.
In constructing residual plots, it is more convenient to use standardized residuals rather than the
actual residuals, which vary widely from case to case. The standardized residual for the ith observation
is defined as
sr YY
si
ii= − ˆ
where
s
YY
nk
ii
i
n
==
−()
−
=
∑
SER
ˆ 2
1
where n = number of observations
 k = number of parameters (which is equal to the number of independent variables plus 1,
assuming there is a constant term in the equation)
In effect, the standardized residual can be thought of as indicating how many standard deviations a
residual is from the assumed residual mean of zero. If the regression assumptions are valid, the stan-
dardized residuals should be randomly distributed and fall primarily in a range between +2 and −2.
One great advantage of using standardized residuals is that the interpretation of a residual plot is the
same for all types of data. There are three basic types of residual plots:
 1. sri versus the fitted Y values ()Y iˆ .
 2. sri versus the independent variables (in multiple regression, there can be one such plot for each
independent variable).
 3. sri versus time (i.e., the sri values are plotted in time order).
The preceding plots are available on some computer packages. However, even when they are not,
they can easily be constructed from the printout of residual values. Since virtually all applications of
regression analysis


e independent variables (in multiple regression, there can be one such plot for each
independent variable).
 3. sri versus time (i.e., the sri values are plotted in time order).
The preceding plots are available on some computer packages. However, even when they are not,
they can easily be constructed from the printout of residual values. Since virtually all applications of
regression analysis in forecasting futures involve time series data, the third type of residual plot will
normally be the most useful.
Figure E.1 provides an example of a residual plot plotted against time. As is readily visible, 2011
and 2014 are outliers. Also note the predominance of negative residuals in the remaining years—a
consequence of the two positive outliers pulling up the regression line. (Remember, the least-squares
procedure will tend to place greater weight on outliers.)
One essential attribute of the residual plot is that it can be applied just as easily when a model con-
tains three or more variables, a situation in which a scatter diagram cannot be constructed. Perhaps
even more importantly, the residual plot can be used to check for autocorrelation. Before turning to
this critical application of the residual plot, it is first necessary to discuss autocorrelation and the most
commonly used method for its detection, the Durbin-Watson (DW) statistic.
651
ANALYZING THE REGRESSION EQUATION
 FIGURE  E.1 Standardized Residuals Plotted Against Time
2000
+2.5
+2.0
+1 .5
+1 .0
+0.5
0
−0.5
−1 .0
−1 .5
−2.0
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
 ■ Autocorrelation Deﬁ ned
 Autocorrelation refers to the situation in which the error terms are correlated. The existence
of autocorrelation indicates that there is a pattern in the data that has not yet been captured
by the explanatory variables. For this reason, the presence of autocorrelation suggests that the
model is still inadequate. Furthermore, it should be recalled that one of the basic assumptions
underlying the statistics of regression analysis is that the error terms are randomly distributed.
If autocorrelation exists, then the standard error of the coefficients and the SER may all be
severely understated. Thus, the normal tests of significance may yield a very distorted picture
of the equation’s precision.
 ■ The Durbin-Watson Statistic as a Measure of Autocorrelation
 Autocorrelation refers to a linear dependency in the error terms. In the simplest case, the error
terms are correlated to the error terms of the preceding period. This type of condition is called fi rst-
order autocorrelation and is refl ected by the DW statistic, which is a standard measure included in the
regression summary printout.
652
Appendix e
Although ideally a test for autocorrelation would be based on the population error terms, these
are unavailable. The DW test is based on the residuals (denoted here by êt) and defined as:
DW
ee
e
tt
t
n
t
t
n=
−() −
=
=
∑
∑
ˆˆ
ˆ
1
2
2
2
2
where êt = residual value in time period t
 êt−1 = residual value in time period immediately preceding period t
The following approximate relationship, however, is more useful for an intuitive explanation of
the Durbin-Watson statistic:
DW r≈−21()
where
r
ee
e
tt
t
n
t
t
n=
⋅ −
=
=
∑
∑
ˆˆ
ˆ
1
2
2
2
If there is no first-order autocorrelation, the positive values for the product ˆˆeett ⋅ −1 will tend to
offset the negative values, and ˆˆeett ⋅∑ −1 should approximate zero. In this case, r will approach zero
and DW will approach 2. If adjacent residuals are positively correlated, then ˆˆeett ⋅ − 1 and ˆˆeett ⋅∑ −1
will tend to be positive. The greater the correlation between adjacent residuals, the more positive
ˆˆeett ⋅∑ −1 will be. In the extreme, ˆˆeett ⋅∑ −1 will approach ˆet
2∑ , causing r to approach 1 and DW to
approach 0. Similarly, if the adjacent residuals are negatively correlated (i.e., positive residuals tend-
ing to follow negative residuals), ˆˆeett ⋅∑ −1 will be negative. If the negative correlation is extreme,
ˆˆeett ⋅∑ −1 will approach −∑ ˆet
2, causing r to approach −1 and DW to approach 4. In summary, the
DW has a possible range between 0 and 4. Values near 2 indicate no first-order autocorrelation, values
significantly below 2 indicate positive autocorrelation, and values significantly above 2 indicate nega-
tive autocorrelation.
Table E.1 contains a list of DW values that can be used to test for autocorrelation at the 0.05 level
of significance. The appropriate values will depend on the number of observations, n, and the number
of independent variables in the regression equation, k. Note that unlike the other tests discussed so
far, there are two values listed for each category. In a test for positive autocorrelation, the interpreta-
tion would be as follows (for negative autocorrelation use 4 − DW instead of DW):
 1. DW < dL Positive autocorrelation exists
 2. DW > dU No positive autocorrelation exists
 3. dL < DW < dU T est is inconclusive
653
ANALYZING THE REGRESSION EQUATION
TAble e.1 The distribution of durbin-Watson d 5 percent Significance points of dl and du
k = 1 k = 2 k = 3 k = 4 k = 5
n d L dU dL dU dL dU dL dU dL dU
15 1.08 1.36 0.95 1.54 0.82 1.75 0.69 1.97 0.56 2.21
16 1.10 1.37 0.98 1.54 0.86 1.73 0.74 1.93 0.62 2.15
17 1.13 1.38 1.02 1.54 0.90 1.71 0.78 1.90 0.67 2.10
18 1.16 1.39 1.05 1.53 0.93 1.69 0.82 1.87 0.71 2.06
19 1.18 1.40 1.08 1.53 0.97 1.68 0.86 1.85 0.75 2.02
20 1.20 1.41 1.10 1.54 1.00 1.68 0.90 1.83 0.79 1.99
21 1.22 1.42 1.13 1.54 1.03 1.67 0.93 1.81 0.83 1.96
22 1.24 1.43 1.15 1.54 1.05 1.66 0.96 1.80 0.86 1.94
23 1.26 1.44 1.17 1.54 1.08 1.66 0.99 1.79 0.90 1.92
24 1.27 1.45 1.19 1.55 1.10 1.66 1.01 1.78 0.93 1.90
25 1.29 1.45 1.21 1.55 1.12 1.66 1.04 1.77 0.95 1.89
26 1.30 1.46 1.22 1.55 1.14 1.65 1.06 1.76 0.98 1.88
27 1.32 1.47 1.24 1.56 1.16 1.65 1.08 1.76 1.01 1.86
28 1.33 1.48 1.26 1.56 1.18 1.65 1.10 1.75 1.03 1.85
29 1.34 1.48 1.27 1.56 1.20 1.65 1.12 1.74 1.05 1.84
30 1.35 1.49 1.28


0.86 1.94
23 1.26 1.44 1.17 1.54 1.08 1.66 0.99 1.79 0.90 1.92
24 1.27 1.45 1.19 1.55 1.10 1.66 1.01 1.78 0.93 1.90
25 1.29 1.45 1.21 1.55 1.12 1.66 1.04 1.77 0.95 1.89
26 1.30 1.46 1.22 1.55 1.14 1.65 1.06 1.76 0.98 1.88
27 1.32 1.47 1.24 1.56 1.16 1.65 1.08 1.76 1.01 1.86
28 1.33 1.48 1.26 1.56 1.18 1.65 1.10 1.75 1.03 1.85
29 1.34 1.48 1.27 1.56 1.20 1.65 1.12 1.74 1.05 1.84
30 1.35 1.49 1.28 1.57 1.21 1.65 1.14 1.74 1.07 1.83
31 1.36 1.50 1.30 1.57 1.23 1.65 1.16 1.74 1.09 1.83
32 1.37 1.50 1.31 1.57 1.24 1.65 1.18 1.73 1.11 1.82
33 1.38 1.51 1.32 1.58 1.26 1.65 1.19 1.73 1.13 1.81
34 1.39 1.51 1.33 1.58 1.27 1.65 1.21 1.73 1.15 1.81
35 1.40 1.52 1.34 1.58 1.28 1.65 1.22 1.73 1.16 1.80
36 1.41 1.52 1.35 1.59 1.29 1.65 1.24 1.73 1.18 1.80
37 1.42 1.53 1.36 1.59 1.31 1.66 1.25 1.72 1.19 1.80
38 1.43 1.54 1.37 1.59 1.32 1.66 1.26 1.72 1.21 1.79
39 1.43 1.54 1.38 1.60 1.33 1.66 1.27 1.72 1.22 1.79
40 1.44 1.54 1.39 1.60 1.34 1.66 1.29 1.72 1.23 1.79
45 1.48 1.57 1.43 1.62 1.38 1.67 1.34 1.72 1.29 1.78
50 1.50 1.59 1.46 1.63 1.42 1.67 1.38 1.72 1.34 1.77
55 1.53 1.60 1.49 1.64 1.45 1.68 1.41 1.72 1.38 1.77
60 1.55 1.62 1.51 1.65 1.48 1.69 1.44 1.73 1.41 1.77
65 1.57 1.63 1.54 1.66 1.50 1.70 1.47 1.73 1.44 1.77
70 1.58 1.64 1.55 1.67 1.52 1.70 1.49 1.74 1.46 1.77
75 1.60 1.65 1.57 1.68 1.54 1.71 1.51 1.74 1.49 1.77
80 1.61 1.66 1.59 1.69 1.56 1.72 1.53 1.74 1.51 1.77
85 1.62 1.67 1.60 1.70 1.57 1.72 1.55 1.75 1.52 1.77
90 1.63 1.68 1.61 1.70 1.59 1.73 1.57 1.75 1.54 1.78
95 1.64 1.69 1.62 1.71 1.60 1.73 1.58 1.75 1.56 1.78
100 1.65 1.69 1.63 1.72 1.61 1.74 1.59 1.76 1.57 1.78
Note: DW values below dL indicate that positive autocorrelation exists; values above dU indicate that no positive autocorrelation exists. DW values
between dL and dU are inconclusive. T o test for negative correlation use 4 − DW instead of DW.
Source: S. Chatterjee and B. Price, Regression Analysis by Example, 3rd ed. (New Y ork, NY: John Wiley & Sons, 1999). Copyright © 1999 by John
Wiley & Sons; reprinted with permission.
654
Appendix e
For example, assume we are testing a regression equation with 18 observations and three vari-
ables. Positive autocorrelation would be indicated if DW < 0.93, no autocorrelation if DW > 1.69, and
the test would be inconclusive if 0.93 < DW < 1.69. The test for negative autocorrelation would be
analogous, using 4 − DW instead of DW.
A routine check of summary statistics for a regression equation should include the DW . A par-
ticularly low or high DW would indicate a definite need for further analysis and model modification.
However, it should be emphasized that even a perfect DW value (2.0) does not guarantee that auto-
correlation does not exist. The DW only tests for first-order autocorrelation. If the interrelationship
between the error terms is more complex, the DW might not pick it up. For this reason, it is also advis-
able to check a residual plot for autocorrelation. Furthermore, as will be illustrated in subsequent sec-
tions, the residual plot can also be used to provide important clues for improving the regression model.
 ■ The Implications of Autocorrelation
The presence of a pattern in the residuals suggests a potential inadequacy in the regression equation.
Specifically, autocorrelation may reflect one of the two following flaws:
 1. The omission of significant explanatory variables in the regression equation.
 2. The use of the linear regression method to describe a nonlinear relationship between the depen-
dent and independent variables.
If the autocorrelation is due to one of these factors, it is clear why autocorrelation is undesirable.
These conditions indicate that a better model can be constructed, either by adding variables to the
equation or by trying different functional relationships. However, even when this is not the case, an
equation that exhibits autocorrelation is still undesirable, because the violation of the assumption that
the error terms are randomly distributed will lead to distortions.
1 For this reason, as a last resort,
transformations designed to remove the autocorrelation should be considered.
T o summarize, the DW and residual plot should be checked for autocorrelation. If residuals are
found to be correlated, the following steps should be taken:
 1. Try to find any significant variables that may have been omitted from the equation.
 2. If all feasible variables have been tried and autocorrelation still exists, experiment to see whether
alternative functional forms (other than the linear form assumed in the regression procedure)
are more appropriate.
 3. If both of the preceding steps are unsuccessful, transformations to remove autocorrelation
might be tried.
1 If autocorrelation exists, the standard regression approach, which is called ordinary least squares (OLS), will still
yield unbiased estimates (i.e., estimates that on average will equal the population parameters). However, the
estimates will no longer be efficient (i.e., they will not be the minimum variance estimates). Even worse, the
standard error estimates of the regression coefficients and the equation as a whole may be severely understated.
Consequently, the true confidence interval may be much wider than suggested, and the regression equation may
be too imprecise to be used for forecasting.
655
ANALYZING THE REGRESSION EQUATION
 The fi rst of these steps will be illustrated by an example in the next section. Methods to address
the second two steps are discussed in the addendum to this appendix.
 ■ Missing Variables and Time Trend
 A pattern in the residual plot (or the presence of autocorrelation) can be viewed as an indica-
tion that signifi cant explanatory variables are missing from the equation. For example, Figure E.2
shows the residual plot for the simple regression model of the average December hog futures price
 during July–November as a function of per capita June–November hog slaughter. Note the obvious
 nonrandom distribution of the residu


(or the presence of autocorrelation) can be viewed as an indica-
tion that signifi cant explanatory variables are missing from the equation. For example, Figure E.2
shows the residual plot for the simple regression model of the average December hog futures price
 during July–November as a function of per capita June–November hog slaughter. Note the obvious
 nonrandom distribution of the residuals: There seems to be a defi nite trending pattern in the residu-
als with large negative values predominating in the earlier years and large positive values predomi-
nating in the later years.
 Given this strong trending pattern in the residuals, we add a time trend as one of the explanatory
variables. A time trend is simply a set of successive integers. Normally, the fi rst observation would be
assigned a value of 1, the second a value of 2, and so on. However, since the regression model is linear,
any set of consecutive integers would serve equally well.
 It is not surprising that the fi tted values of the original equation tend to be too high in the earlier
years and too low in the later years since our model used nominal rather than defl ated prices. The
reader may well wonder why we didn’t fi rst change the model by using defl ated prices instead of
adding a time trend. In fact, this alternative approach is entirley reasonable as the fi rst change to try,
 FIGURE  E.2 Standardized Residuals for Average Price of December Hog Futures
(July–November) vs. June–November Hog Slaughter
 Standardized Residuals for Average Price of December Hog Futures
−2.0
−1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
656
Appendix e
and we did run this model (not shown), but the results were substantially inferior to the model that
added a time trend.
Table E.2 compares the summary statistics of this two-independent-variable regression equation
with those of the original one-independent-variable model. Note the dramatic improvement in all the
summary statistics and the significance of the time trend as reflected by its t statistic. In fact, the time
trend is statistically even more significant in explaining hog prices than hog slaughter! In addition,
the strong trend evident in the residual plot for the original simple regression (Figure E.2) has been
eliminated in the residual plot for the new equation (Figure E.3). Although the trend in the residuals
has been eliminated, Figure E.3 still exhibits a non-random pattern. Specifically, the residuals now
conform to a broad “U” pattern, with positive residuals predominating in the early and late years and
negative residuals predominating in the middle years. The existence of this pattern suggests other
significant variables are still missing from the equation.
Next we add per capita broiler slaughter to the model, since poultry is an important competi-
tive meat to pork. Table E.3 compares the key statistics for this new equation (Model 3) with the
 corresponding values for the first two models. As can be seen, the addition of poultry slaughter
 provides a large improvement in all the key statistics. For example, the corrected R2 jumps from
0.66 in Model 2 to 0.82 in Model 3. Moreover, not only is the t statistic for broiler slaughter highly
significant but the addition of this variable also increases the t statistics for the other explanantory
variables (hog slaughter and trend). The addition of broiler slaughter to the equation also eliminates
the pattern in the residuals. As can be seen in Figure E.4, which shows the residual plot corresponding
to Model 3, the scatter of residuals now seems random.
Achieving a random residual plot doesn’t necessarily mean the model is complete. It may well
be possible to further improve the model by adding other variables. Model 4 in Table E.3 illustrates
TAble e.2 Regression Summary Statistics for Hog-price-Forecasting Models
Statistic
Model 1: Hog price vs. per Capita
Hog Slaughter
Model 2: Hog price vs. per Capita
Hog Slaughter and Trend
R2 0.21 0.66
CR2 0.20 0.64
SER 13.95 9.30
%SER 27.2 18.16
F 11.72 40.62
t-stat (constant) 5.19 4.57
t-stat (hog slaughter) –3.42 –3.12
t-stat (trend) NA 7.41
t-stat (broiler slaughter) NA NA
t-stat (cattle slaughter) NA NA
657
ANALYZING THE REGRESSION EQUATION
 FIGURE  E.3 Standardized Residuals for Average Price of December Hog Futures
(July– November) vs. June–November Hog Slaughter, and Trend
−3.0
−2.5
−2.0
−1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
 TAble e.3 Regression Summary Statistics for Hog-price-Forecasting Models
Statistic
Model 1: Hog
price vs. per Capita
Hog Slaughter
Model 2: Hog
price vs. per Capita
Hog Slaughter and
Trend
Model 3: Hog price
vs. per Capita Hog
Slaughter, broiler
Slaughter, and Trend
Model 4: Hog price vs.
per Capita Hog Slaughter,
broiler Slaughter, Cattle
Slaughter, and Trend
R 2 0.21 0.66 0.84 0.85
CR 2 0.20 0.64 0.82 0.84
SER 13.95 9.30 6.53 6.24
%SER 27.2 18.16 12.76 12.18
F 11.72 40.62 69.51 58.40
t -stat (constant) 5.19 4.57 8.45 6.02
t -stat (hog slaughter) –3.42 –3.12 –4.39 –5.11
t -stat (trend) NA 7.41 10.29 5.88
t -stat (broiler slaughter) NA NA –6.64 –7.23
t -stat (cattle slaughter) NA NA NA –2.22
658APPENDIX E
 FIGURE  E.4 Standardized Residuals for Average Price of December Hog Futures
(July– November) vs. June–November Hog Slaughter, Broiler Slaughter, and Trend
−3.0
−2.0
−1.0
0.0
1.0
2.0
3.0
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
one such attempt: adding per capita cattle slaughter to the model on the premise that beef is another
competitive meat to pork. The t statistic for cattle slaughter is statistically signifi cant and adding this
variable modestly increases the corrected R 2 and low


0
1.0
2.0
3.0
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
one such attempt: adding per capita cattle slaughter to the model on the premise that beef is another
competitive meat to pork. The t statistic for cattle slaughter is statistically signifi cant and adding this
variable modestly increases the corrected R 2 and lowers the SER.
 ■ Dummy Variables
 In Appendix A we derived a regression equation for forecasting June–November hog slaughter from
the prior December–May pig crop. Consider what happens when we attempt to make the equation
more general by forecasting hog slaughter during a six-month period from the pig crop of the previ-
ous six-month period. In this case, half the observations are those of the original equation, while the
other half relate December–May slaughter to the June–November pig crop. Figure E.5 illustrates
the residual plot for this equation. W e have used two diff erent symbols to distinguish between the
residuals for June–November slaughter and the residuals for December–May slaughter. Note the
striking pattern of the predominance of positive residuals for June–November slaughter and nega-
tive residuals for December–May slaughter. As Figure E.5 dramatically indicates, our equation is
missing some important information: the seasonal period of the slaughter forecast. Clearly, we want
our equation to distinguish between the two periods. In other words, it is necessary to include a
seasonal indicator.
659
ANALYZING THE REGRESSION EQUATION
 A simple method for handling such a situation would be to add a dummy variable to the equation,
which has a value of 1 for one season and a value of 0 for the other season. The regression equation
adding a dummy variable could be written as
 HS ab PC cS=+ +
where HS = hog slaughter
PC = pig crop
S = dummy variable, which equals 0 during December–May and 1 during June–November
 The dummy variable can be thought of as a switch that is set to off (0) during the base period
(December–May) and on (1) during June–November. The dummy variable will have the eff ect of
shifting the intercept by an amount c for the June–November observations. Note that this adjustment
will be exactly equivalent to fi nding two separate equations with the same slope, one for each period.
That is, HS = a + bPC + cS for all periods is equivalent to:
 HS ab PC
HS ab PC
=+
=+
1
2
forD ecember-Ma ys laughte r
forJ une-November s slaughte r
where a 2 = a 1 + c
 FIGURE  E.5 Standardized Residuals for Hog Slaughter vs. Previous Six-Month Pig Crop
−2.0
−1.5
−1.0
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
2016
December-May HS
June-November HS
660
Appendix e
Typically, most users of regression analysis will only employ a dummy variable to shift the inter-
cept, while the slope is assumed to remain constant from period to period. However, in most instances
there is no reason to impose an a priori restriction that the slopes be equal in different periods. Rather,
it seems preferable to begin by using dummy variables for both the intercept and the slopes.
2 Once
this full version of dummy variables is run, we can check the t statistics to see which of the dummy
variables are significant and then choose the appropriate model accordingly. Thus, in our example,
we would begin with:
HS ab PC cS dS PC=+ ++ ⋅⋅
where S = 0 during December–May
 S = 1 during June–November
The form of the equation used will depend on which of the dummy variables proves significant.
Some examples:
 1. If neither c or d is significant, we would use:
HS ab PC=+
 2. If only c is significant, we would use:
HS ab PC cS=+ +
 3. If both c and d are significant, we would use the full-version equation:
HS ab PC cS dS PC=+ ++ ⋅⋅
2 There are two important exceptions: (1) When one of the periods contains only a few observations, the slope
estimate for this period might be unreliable, and it would be better to pool the data in terms of assuming a
common slope coefficient for all observations. For example, consider an annual price-forecasting model with
15 observations, three of which coincided with a government program that distorted normal market behavior.
In this case, we would definitely only use the dummy variable for the constant term (with the aforementioned
three years having a dummy variable value equal to 1), thereby implicitly imposing the restriction of a common
slope. The reason for this is that a slope estimate based on only three observations would not be very reliable. This
example illustrates one of the advantages of using dummy variables, as opposed to separate equations for each
set of observations. (2) When the number of all possible dummy variables is large compared with the number of
observations, it may be desirable to conserve degrees of freedom by limiting the number of dummy variables.
661
ANALYZING THE REGRESSION EQUATION
Note that in this last case, when both c and d are significant, the resulting equation is equivalent to
the following two separate equations for each period:3
HS = a + bPC for December–May
HS = (a + c) + (b + d)PC for June–November
Why, then, do we not just run separate equations for each period? There are several reasons:
 1. By pooling the data, we increase the number of degrees of freedom and add to the statistical
reliability of the equation.
 2. W e do not know beforehand which, if any, of the dummy variables will be significant. The
single-equation approach will allow us to eliminate the dummy variables that appear insignifi-
cant, thereby providing a better model. In contrast, the two-equation approach is equivalent to
automatically assuming that all the dummy variables are significant.
 3. In terms of the various tasks of checking alternative models, testing for significance, and fore-
casting, it is more convenient to have a single equation that is applicable to all periods than a
separate equation


pear insignifi-
cant, thereby providing a better model. In contrast, the two-equation approach is equivalent to
automatically assuming that all the dummy variables are significant.
 3. In terms of the various tasks of checking alternative models, testing for significance, and fore-
casting, it is more convenient to have a single equation that is applicable to all periods than a
separate equation for each period.
 4. As mentioned in footnote 2, there are times when it is definitely preferable to impose slope
restrictions—an approach that requires the use of dummy variables.
Since in our example of hog slaughter versus the prior six-month pig crop the dummy variable for
the slope is statistically significant, we use the full form of the equation:
HS ab PC cS dS PC=+ ++ ⋅⋅
Figure E.6 shows the residual plot for the regression equation that adds dummy variables for the
slope and intercept. Note that the positive bias for June–November residuals and the negative bias for
December–May residuals has been eliminated.
The failure to include dummy variables when they are appropriate will bias the regression coef-
ficient estimates. In Figure E.7 we provide a hypothetical example where the points associated with
two different periods are best described by best-fit lines with different constant terms. Note how the
slope of a single regression equation line without inclusion of dummy variables is biased by the failure
to distinguish between the two periods.
Although our example involved only two periods (one period other than the base period), the
dummy variable approach can be extended to more period divisions. For example, if we were using
a quarterly model, there would be one dummy variable for each quarter other than the base quarter.
Ya bX cS cS cS dS Xe SX fS X=+ ++ ++ ⋅⋅ +⋅ ⋅+ ⋅⋅11 22 33 12 3
3 Although the intercept and slopes will be identical in the one- and two-equation versions, there is a minor
technical difference between the two models. The single equation implicitly assumes a common variance for
all periods, while the two-equation version allows for different variances in each period. This difference could
theoretically affect the various tests of significance.
662APPENDIX E
 FIGURE  E.6 Standardized Residuals for Hog Slaughter vs. Previous Six-Month Pig Crop after
Including Dummy Variables
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
2014
2016
December-May HS
June-November HS
 FIGURE  E.7 Bias in Regression Line Due to Omission of Dummy Variables
Fitted line without use of
dummy variables
Two equations implied by equation that
includes a dummy variable for constant term
663
ANALYZING THE REGRESSION EQUATION
where S1 = dummy variable for the first quarter
 S2 = dummy variable for the second quarter
 S3 = dummy variable for the third quarter
Note that the number of dummy variables is always equal to one less than the number of periods,
since the base period conditions, assumed to be the fourth quarter in our example, are captured by
the original constant and regression coefficient.4
If there are two independent variables, the full-version equation would be
Ya bX bX cS cS cS dS X
dS Xe SX e
=+ ++ ++ +⋅ ⋅
⋅⋅ +⋅ ⋅++
11 22 11 22 33 11 1
21 21 21 2 ⋅⋅ ⋅+ ⋅⋅ +⋅ ⋅SX fS Xf SX22 13 12 32
Note that:
b values are regression coefficients for regular independent variables.
c values are regression coefficients for dummy constants.
d values are regression coefficients for dummy slope for the first period (S
1).
e values are regression coefficients for dummy slope for the second period (S2).
f values are regression coefficients for dummy slope for the third period (S3).
As should be quite apparent, when the number of periods is increased, the number of dummy
variables increases like rabbits. Since the researcher might wish to avoid beginning with an equa-
tion that contains a large number of dummy variables, she might prefer to only include constant
dummy variable terms in the starting equation and then experiment with the addition of selected
slope dummy variable terms if she believes that her initial equation needs improvement.
 ■ Multicollinearity
The reader may recall that the extension to the multiple regression model required the addi-
tional assumption that the independent variables be linearly independent. Multicollinearity is a
term used to describe the presence of significant correlation between two or more independent
variables.
T o see why multicollinearity is a problem, consider a hog-slaughter-forecasting model that includes
both the pig crop during the prior six-month period and the number of market hogs at the start
of the period as explanatory variables. In this case, the independent variables would be extremely
highly correlated, that is, large market hog figures would coincide with large pig-crop numbers. As
illustrated in Figure E.8, a three-dimensional representation is really unnecessary for this model, as
4 In fact, including a dummy variable for the base period would actually result in perfect multicollinearity—a
totally undesirable situation (see next section).
664APPENDIX E demonstrated by the proximity of the points to a straight line. Actually, either the X 1, Y plane or the
X 2, Y plane alone would have been adequate. The fi rst plane would be a two-dimensional representa-
tion of the relationship between hog slaughter and the pig crop, and the second, a two-dimensional
representation of the relationship between hog slaughter and market hogs. In eff ect, the inclusion of
both the pig crop and market hogs forces the use of a three-dimensional model to represent a relation-
ship that can be adequately described by two dimensions.
 The problem lies not in the fact that multicollinearity implies the inclusion of superfl uous
information, but rather that this redundancy can severely aff ect the regression equation’s reli-
ability. Multicollinearity is a perfect example of the phrase “more is les


hogs forces the use of a three-dimensional model to represent a relation-
ship that can be adequately described by two dimensions.
 The problem lies not in the fact that multicollinearity implies the inclusion of superfl uous
information, but rather that this redundancy can severely aff ect the regression equation’s reli-
ability. Multicollinearity is a perfect example of the phrase “more is less.” As can be seen in Figure
 E.8 , when multicollinearity is present, there may be very divergent planes that closely approxi-
mate the fi t of points. For any given set of observations, the regression procedure will choose one
plane that best fi ts the observations. However, the real problem in multicollinearity lies in the
fact that if the observations were only slightly altered, a totally diff erent plane might be chosen
as the best fi t. Thus, if multicollinearity exists, the regression coeffi cients are no longer reliable
indicators of how the dependent variable will change when each of the independent variables is
changed (while all the other independent variables are held constant). This fact will be refl ected
by high standard errors, and hence low t statistics, for the regression coeffi cients of highly cor-
related explanatory variables.
 FIGURE  E.8 Multicollinearity
 Source: Adapted from T . H. W onnacott and R. J. W onnacott, Econometrics , John Wiley & Sons,
New Y ork, 1980.
Y = hog slaughter
X1 = pig crop
X2 = market hogs
665
ANALYZING THE REGRESSION EQUATION
What about the reliability of the equation in forecasting? If the values of the independent variables
for the forecast period lie in the neighborhood of past observations, then the multicollinear model can
still provide adequate forecasts. This situation describes the preceding example, since presumably the
pig crop and market hogs will continue to remain highly correlated. In other circumstances, however,
if the two correlated independent variables cease to be correlated in the future, then the forecast
provided by the multicollinear equation could be distorted because the model is only valid for points
in the neighborhood of past observations. At other locations, there is no historical evidence to provide
any clues regarding the expected relationship between the variables. In geometric terms, all planes
passing through a line provide accurate forecasts in the vicinity of the line, but drastically different
projections at points removed from the line (Figure E.8).
T o summarize, there are two major drawbacks to a multicollinear equation:
 1. The regression coefficients lose their meaning (i.e., are no longer statistically reliable).
 2. If the equation is used for forecasts in which the independent variables do not lie in the neigh-
borhood of past observations, the projection could be severely distorted.
Clearly, then, it is always desirable to avoid multicollinearity. The presence of multicollinearity can
be detected in a number of ways:
 1. Check the regression coefficients. The regression coefficients of an equation can provide a
number of clues indicating that multicollinearity is present:
a.
 Low t statistics for coefficients that were expected to be highly significant.
b. In more extreme cases, a regression coefficient sign that may actually be counter to theoreti-
cal expectations.
c.
 Large changes in the coefficient values when variables are added or deleted from the equation.
d. Large changes in coefficient values when data points are added or deleted from the equation.
Any of these patterns would suggest that the independent variables should be examined for signs
of correlation.
 2. Compare the independent variables. Sometimes common sense will dictate when the
independent variables are likely to be correlated. By being aware of the problem, one can often
avoid multicollinearity by carefully selecting the independent variables. For example, if the
researcher thought that gross national product (GNP) and disposable income might help explain
the variation of the dependent variable, she would use either one, or try both successively, but
she would not include them in the same equation simultaneously. Beyond intuition, one can
check for correlation between the independent variables statistically. High absolute values of
the correlation coefficients
5 between any two independent variables would indicate a potential
5 The correlation coefficient, denoted by the symbol r, reflects the degree of relationship between two variables
and can range between −1 and +1. Values close to +1 indicate a strong positive relationship, while values close to
−1 indicate a strong inverse relationship. If r is close to 0, it means that there is little, if any, correlation between
the two variables. The square of the correlation coefficient is equal to the r
2 of the simple regression equation in
which one of the variables is a dependent variable and the other an independent variable.
666
Appendix e
multicollinearity problem. The correlation matrix—an output feature in some software pack-
ages—offers a summary array of all the paired correlation values.
What should be done if multicollinearity is discovered in an equation? One solution is simply to
delete one of the correlated independent variables.
 ■ Addendum: Advanced Topics
Transformations to Achieve linearity6
Perhaps the most basic assumption in a regression analysis is that the relationship between the
dependent and independent variables is approximately linear. If, in fact, the relationship is deci-
sively nonlinear, the error terms might appear to be correlated. For example, consider what hap -
pens when we try to fit a regression line to the scatter points in Figure E.9a. Forcing these points
into a linear fit would result in the residual pattern illustrated in Figure E.9b, in which the residuals
would tend to be positive at high and low values of the independent variable X and negative in the
middle range of values. (In Figure E.9b, the sta


rrelated. For example, consider what hap -
pens when we try to fit a regression line to the scatter points in Figure E.9a. Forcing these points
into a linear fit would result in the residual pattern illustrated in Figure E.9b, in which the residuals
would tend to be positive at high and low values of the independent variable X and negative in the
middle range of values. (In Figure E.9b, the standardized residuals are plotted against the indepen-
dent variable not time.)
Fortunately, many nonlinear relationships can be transformed into linear equations. For example,
the scatter of points in Figure E.9a suggests a hyperbolic function, or an equation of the general form
Ya b
Xc=+ +
This can be transformed into a linear relationship by setting
X Xc′= +
1
then
Ya bX=+ ′
In this form, the equation can be solved in straightforward fashion using ordinary least squares
(OLS), the standard regression procedure. T o get a specific forecast for Y, one would merely plug in
the value 1/(X + c) for X′. For example, if a = 2, b = 16, c = 4, and X = 4, the forecast for Y is 4.
6 Although still involving nothing more mathematically complex than algebra, the remainder of this appendix
covers material that is somewhat more advanced.
667
ANALYZING THE REGRESSION EQUATION
 FIGURE  E.9 Distortion in Applying Linear Regression to Nonlinear Function
Y
X
0
(a) (b)
Standardized residuals
X
 Many other types of functions can be transformed into linear equations. Let us consider a few
more examples:
 1. Y = a + b 1 X + b 2 X 2 + b 3 X 3
 Let X 1 = X ; X 2 = X 2; X 3 = X 3; then
 Y = a + b 1 X 1 + b 2 X 2 + b 3 X 3
 This is a linear equation and OLS can be applied. Note that although the independent vari-
ables are related, the relationship is nonlinear, so that the regression assumption regarding linear
independence among the explanatory variables is not violated.
 2. Y = ae bX
 Taking the natural logarithm of both sides:
 ln lnYa bX=+
 Let Y ′ = ln Y ; a ′ = ln a ; then
 Ya bX′= ′+
 This is a linear equation and OLS can be applied. Note that in this case, plugging a value for
 X into the derived regression equation will yield a forecast for ln Y . T o get a forecast for Y , it
would be necessary to fi nd the antilog value. Figure E.10 illustrates the shapes of the function
 Y = ae bX for diff erent values of b.
668APPENDIX E
 FIGURE  E.10 Y = ae bX
 Source: S. Chatterjee and B. Price, Regression Analysis by Example, 3rd ed. (New Y ork, NY: John Wiley
& Sons 1999). Copyright © 1999 by John Wiley & Sons; reprinted with permission.
(b > 0)
1/b
(a)
x
y
ae
0
(b < 0)
1/|b|
(b)
x
y
a/e
0
 3. Ya Xb=⋅
 Taking logs of both sides:
logl og logYa bX=+
 Let Y ′ = log Y ; a ′ = log a ; X ′ = log X ; then
Ya bX′= ′+ ′
 This is a linear equation and OLS can be applied. Here we would plug the value for log X , not
X , into the regression equation to get a forecast of log Y . T o get a forecast for Y , it would then
be necessary to fi nd the antilog value. Figure E.11 illustrates the shape of the function Y = aX b
for diff erent values of a and b .
 If a residual plot still refl ects autocorrelation after all feasible variables have been tried, the pos-
sibility of nonlinearity should be considered. In the simple regression case, a scatter diagram can
be constructed in order to check whether a linear assumption is warranted or whether another
functional form is more appropriate, just as Figure E.9 a suggested the equation form Y = a + b/ ( X +
c ). In a multiple regression, if nonlinearity is expected for one of the independent variables, a regres-
sion could be run using only the other independent variables. The residuals of this equation would
then be plotted against the unused independent variable. The presence of any nonlinearity would be
apparent in the resulting scatter diagram.
669
ANALYZING THE REGRESSION EQUATION
 FIGURE  E.11 Y = aX b
 Source: S. Chatterjee and B. Price, Regression Analysis by Example , 3rd ed. (New Y ork, NY:
John Wiley & Sons, 1999). Copyright © 1999 by John Wiley & Sons; reprinted with permission.
(a, b, x all > 0)
x
a
1
0
b > 1
(a)
b = 1
0 < b < 1
y
(a, x > 0, b < 0)
x
a
1
0
(b)
−1 < b < 0
b = −1
b < −1
y
 Transformation to Remove Autocorrelation
 The simplest assumption one can make about autocorrelation is that a current period’s error
term will be equal to the previous period’s error term plus a random disturbance. This can be
expressed as
ee
tt t=+ −1 υ ,
where υ t = a random disturbance term.
 Since Y t = α + βX t + e t and Y t −1 = α + βX t −1 + e t −1 , then
YY XXtt tt t−= −+−−11 βυ()
 Let YY Ytt t
* =− −1 and XX Xtt t
* =− −1 ; then
 YXtt t
** =+βυ
 For a k -variable multiple regression equation, these steps would yield
 YX XXtt tk kt t
** **=+ ++ +ββ βυ11 22 
 Since by defi nition υ t is randomly distributed, OLS can now be applied to this equation.
670
Appendix e
The preceding method, which is perhaps the most commonly used transformation for removing
autocorrelation, is known as first differences. In effect, the first difference regression equation states that
the change in Y will be linearly dependent on the change in X. Equations of this type will tend to have
much lower R2 values. This is only to be expected, since forecasting the change from one period to the
next is much more difficult than forecasting the level. Once again, consider the following daytrading
price-forecasting model:
Pa bPtt =+ −1
where Pt = closing price on day t
 Pt − 1 = closing price on day t − 1
Such an equation would have an extremely high R2 since it would give us a close approximation of
the price level for a given day. However, it would be useless in forecasting the change in price from
day to day. The model
Pa bXtt
** =+
where PP Ptt t
* =− −1
XX Xtt t
* =− −1
in which Xt is some explanatory variable the value of which is known before day t, would be far pref-
erable, even if its R2 value were low (e.g., R2 = 0.30).
The first difference approach is easy to


ose approximation of
the price level for a given day. However, it would be useless in forecasting the change in price from
day to day. The model
Pa bXtt
** =+
where PP Ptt t
* =− −1
XX Xtt t
* =− −1
in which Xt is some explanatory variable the value of which is known before day t, would be far pref-
erable, even if its R2 value were low (e.g., R2 = 0.30).
The first difference approach is easy to use, but it does involve an extreme simplifying assumption
regarding the nature of the autocorrelation. A more realistic assumption would be
eett t=+ −ρυ 1
where | ρ | < 1. Note that the larger the value of ρ, the more the error term in a given period will
be dependent upon the previous period’s error term. A generalized transformation is analogous to
the first difference transformation:
YX e
YX e
tt t
tt t
=+ +
=+ +−− −
αβ
αβ

 11 1
If we multiply the equation for Yt − 1 by ρ
ρρ αρ βρYX ett t−− −=+ +11 1
 Thus, Yt − ρYt − 1 = α(1 − ρ) + β(Xt − ρXt − 1) + υt.
Let YY Ytt t
* =− −ρ 1 and XX Xtt t
* =− −ρ 1.
Then YXtt t
** ()=− ++αρ βυ1 .
671
ANALYZING THE REGRESSION EQUATION
For a k-variable equation, these steps would yield
YX XXtt kk tt
** **()=− ++ ++ +αρ ββ βυ1 11 22 
Since by definition υ t is randomly distributed, OLS can once again be used. The only problem
with this procedure is that we do not know the value of ρ. W e very briefly describe two approaches
for estimating ρ.
 1. The Hildreth-lu procedure. This procedure specifies a set of spaced values for ρ. If positive
autocorrelation is assumed, these values might be 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9,
1.0. A regression is then run for the transformed equation:
YX XXtt tk kt
** **()=− ++ ++αρ ββ β1 11 22 
using each of the specified values. The procedure will select the equation that results in the low-
est SER. If desired, the process can be repeated using a closer spacing of ρ values in the vicinity
of the ρ selected in the initial step.
 2. The Cochrane-Orcutt procedure. This iterative procedure estimates a ρ value from the
residuals of the original equation, and a regression is then run on the transformed equation
using this estimate of ρ. If the resulting equation still indicates autocorrelation, the process is
repeated using the residuals of the new equation.
 ■ Heteroscedasticity
One of the assumptions that justifies the use of ordinary least squares (OLS) is that the error terms
are homoscedastistic, that is, they have an approximate constant variance. When this condition is not
met, the problem is called heteroscedasticity. Figure E.12 illustrates a case of heteroscedasticity. Note
that the relationship between the dependent and independent variables becomes increasingly variable
as X increases, resulting in higher absolute residual values at higher values of X. The wider variability
between the dependent and independent variables in a given region will make the resulting regression
equation less reliable.
Weighted least squares (WLS) is a method used to circumvent this problem. For the relationship
depicted in Figure E.12, the WLS approach would give greater weight to the observations for lower
values of X, since these offer a more precise indication of the location of the true regression line.
Rather than describe the WLS procedure, suffice it to say that there is a simpler approach using a
transformation that achieves exactly equivalent results. This transformation assumes that the standard
deviation of the error terms is proportional to the independent variable. Specifically,
σii kX=
672APPENDIX E
where σ i = standard deviation of the error terms ( e i ). Starting with the standard regression equation
YX eii i=+ +αβ
we divide by X i ,

Y
X
a
X
e
X
i
ii
i
i
=+ + β
 The standard deviation of e i / X i . is equal to the standard deviation of e i divided by X i . Since the
standard deviation of e i is σ i , which equals kX i , the standard deviation of e i /X i = k , a constant. Thus,
this transformation removes the heteroscedasticity of the original equation. Now if we let
 ′= ′ = ′ = ′ = ′ =Y Y
X X X e e
Xi
i
i
i
i
i
i
1 αβ βα and
then
 ′= ′+ ′′ + ′YX eii iαβ
 This equation can be solved by using OLS, yielding
′=+ ′Ya bXii
where a is an estimator of β in the original equation and b is an estimator of α in the original equation.
 FIGURE  E.12 Heteroscedasticity
Y
X
0
(a) (b)
Standardized residuals
X
673
Appendix F
I remember the rage I used to feel when a prediction went awry. I could have shouted at the
subjects of my experiments, “Behave, damn you, behave as you ought!” Eventually I realized
that the subjects were always right. It was I who was wrong. I had made a bad prediction.
—Burrhus Frederic Skinner
 ■ Determining the Dependent Variable
The title of this section might sound trivial. After all, the dependent variable is what we wish to fore-
cast. However, in a price-forecasting equation, the selection of a dependent variable is by no means
obvious. The following choices must be made:
 1. Should the price be stated in nominal or deflated terms?
 2. Should the price be based on cash or futures?
Practical
Considerations
in Applying
Regression Analysis
674
Appendix F
 3. If the price is based on futures, should it be based on a nearest futures price series or a single
contract?
 4. Should the price represent the entire season or only a specified segment of the season?
The answer to question 1 would typically be deflated prices, unless a trend variable is included in
the equation, in which case nominal prices may be a better choice. If, however, the equation does not
include a trend variable, then the use of nominal prices implicitly assumes that equivalent fundamental
conditions in two widely spaced years should result in approximately equal price levels. Obviously, this
assumption is wrong. All else being equal, inflation will result in considerably higher prices in the more
recent year. The subject of adjusting prices for inflation is covered in greater detail in Cha