33.Properties of the Multiple Regression Coefficients: unbiasedness, efficiency, precision, consistency.

PROPERTIES OF THE OLS REGRESSION COEFFICIENTS

MODEL SPECIFICATION

Define the true regression model as:

y_i = _ + _X_i + u_i

Where y_i is composed of (i) a systematic portion _ + _X_i whose specification is guided by economic theory and (ii) a random component given by the disturbance term u_i

Given the Gauss-Markov condition E{u_i} = 0 then E{y_i} = _ + _X_i

Define the estimated regression line as:

= b_ + b_X_i

where b_ and b_ are the OLS estimates of the true regression coefficients _ and _ given a sample of n observations on X and Y:

b_= b_ =

b_

Linearity of the OLS estimators.

We can note that both b_ and b_ are linear estimators since they can be written as linear functions of the Y_i’s (i.e. in the form w_iY_i where the weights, w_i, depend on the (fixed) values taken by the explanatory variable)

EXPECTED VALUES OF THE OLS ESTIMATORS

Each regression coefficient can be decomposed or broken down into (i) a fixed component equal to the value of the true regression parameter (ii) a random component dependent on u which is responsible for its variations around the central tendency:

b_ ; b_

We can then show that:

E{b_} = _ if Gauss-Markov conditions 1 and 4 hold

E{b_} = _ if Gauss-Markov condition 4 holds

Unbiasedness of the OLS estimators.

An estimator is said to be unbiased if the expected value of the estimator is equal to the population characteristic. Sample estimates derived from an unbiased estimator will be accurate on average, though individual estimates (obtained from particular samples) will only equal the true value by coincidence.

precision of the ols estimators

The population variances of the OLS estimators are:

pop.var{b_} ; pop.var{b_} ;

where = E[ is the population variance of the disturbance term. Both pop.var{b_} and pop.var{b_} are (i) directly proportional to and (ii) inversely proportional to n and Var(X)

The population variance of the disturbance term will in general be unknown. An unbiased estimator of is given by:

where e_i = Y_i (b_ + b_X_i) are the residuals from the estimated regression, and nk is the number of degrees of freedom (n is the sample size and k is the number of coefficients in the regression (which is equal to 2 in the simple bivariate model)). The square root of this estimator s_u is known as the (sample) Standard Error of the Regression (S.E.R) and provides an estimator of the typical size of the disturbance term in the regression model.

Estimators for the standard errors of b_ and b_ are therefore given as:

s.e.{b_}

s.e.{b_} ;

The standard error of a regression coefficient is a measure of the typical deviation of a sample estimate of the coefficient from the true parameter value (i.e. of the typical size of the estimation error).

Efficiency of the OLS estimators.

One unbiased estimator is said to be more efficient than another if it has the smaller variance of the two estimators. The more efficient an estimator is the more likely it is that an estimate obtained using the estimator from a particular sample will be close to the value of the population characteristic. The minimum variance unbiased estimator has the smallest variance of any possible unbiased estimator (of its type).

Consistency of the OLS estimators

An estimator is said to be consistent if the probability limit (plim) of the estimator is equal to the true value of the population characteristic. A consistent estimator is bound to give an accurate estimate if the sample size is large enough, regardless of the actual observations in the sample.