31.Autocorrelation. Possible Causes of Autocorrelation. The Durbin–Watson Test.

Autocorrelation is a presence of series correlation between error terms. It describes a correlation between values of the process at different points in time, as a function of the two times or off the time difference. Positive autocorrelation, when on average if the residual (u) at time “t−1” is positive, the residual at time t is likely to be also positive; similarly, if the residual at t − 1 is negative, the residual at t is also likely to be negative. Negative autocorrelation, indicated by an alternating pattern in the residuals, when on average if the residual at time t − 1 is positive, the residual at time t is likely to be negative; similarly, if the residual at t − 1 is negative, the residual at t is likely to be positive.

Autocorrelation is used in security analysis. For example, if you know a stock historically has a high positive autocorrelation value and you witnessed the stock making solid gains over the past several days, you might reasonably expect the movements over the upcoming several days (the leading time series) to match those of the lagging time series and to move upwards.

In order to test for autocorrelation, it is necessary to investigate whether any relationships exist between the current value of the residuals and the previous values of the residuals. The most common diagnostic for presence of autocorrelation based on the estimated residuals is the Durbin and Watson test.

If there is positive autocorrelation in the errors, this difference in the numerator will be relatively small, while if there is negative autocorrelation, with the sign of the error changing very frequently, the numerator will be relatively large. The range of values between DW lies between 0 and 4. The closer it is to 0 then we get positive autocorrelation (cannot use OLS model). The closer it is to 4 we get negative autocorrelation (can use OLS model). The closer it is to 2 we get no autocorrelation. According to the 3d Gauss-Markov condition U should be distributed independently, so there should be no autocorrelation.

32.Multiple Regression Analysis. Derivation of the Multiple Regression Coefficients.

Derivation of OLS estimators

As in the simple regression case, we choose the values of the regression coefficient to make the fit as goog as possible in the hope that we will obtaine the most satisfactory estimates of the unknown true parameters. Our definition of goodness of fit is the minimization of RSS, the sum of squares of the residuals: RSS= , where ei is the residual in observation I, the difference between the actual value Yi in that observation and the value predicted by the regression equation:

= b₁ + b₂X_2i  b₃X_3i  ........  b_kX_ki;

where b₁, b₂ .... b_k are estimates of the true regression coefficients ₁, ₂,......._k, given a sample of n observations on Y and all the X’s; defines the predicted or estimated value of Y_i given the values X_2i, X_3i, .......X_ki, and

e_i = Y_i  = Y_i  b₁  b₂X_2i  b₃X_3i  ........  b_kX_ki are the residuals from the estimated regression. Note that the X variables now have two subscripts. The first identifies the X and the second identifies the observations.

The OLS estimators are obtained by minimising the residual sum of squares (RSS):

RSS= =

The k first order conditions for this problem are known as the normal equations for the regression coefficients:

RSS/b₁ = = 0 (1)

RSS /b₂ = = 0 (2)

RSS /b₃ = = 0 (3)

 

RSS /b_k = = 0 (k)

We can derive the OLS estimator for b₁ from (1) as:

b₁=

where b₁ is the predicted value of Y when the values of all the explanatory variables are set equal to zero.

We can the proceed to derive estimators for the k-1 unknown slope coefficients ₂, ₃ ,..._k from the remaining k-1 simultaneous equations. These estimators are complicated expressions in which each b_j depends in general on the sample values of the dependent variable and all of the explanatory variables.

Slope estimators when k-1 = 2

You well never need to know the formulae for the slope estimators for use in practical work since the computer will perform all the necessary calculations. Nevertheless some insights into the general nature of the slope estimators in multiple regression can be obtained by examining the estimators for the particular case where there are two explanatory variables (i.e. k-1 =2).

Thus when k-1 =2 we obtain: and a parallel expression for b₃ can be obtained by exchanging X₂ and X₃.

The estimator b₂ depends not only on the observed values of X₂ and Y but also on those of X₃. Thus the slope coefficient on X₂ will not generally be the same as that obtained from a simple regression of Y on a constant and X₂. Moreover we can not generally obtain the full set of regression coefficients by performing two simple regressions, one of Y on a constant and X₂ and the other of Y on a constant and X₃, and somehow combining the results.

First, the principles behind the derivation of the regression coefficient are the same for multiple regression as for simple regression. Second, the expression, however, are different and so you should not try to use expressions derived for simple regression in a multiple regression context.