Posted by: geico December 11, 2007
statistic help
Login in to Rate this Post:     0       ?        

The basic assumption behind increasing the independent variables or predictors is to improve F statistics and more predictors in the model lowers the bias of predictions but increases the variance. So there is always a trade-off and when we underfit the model (say ignore important predictors) we deposit the variation accounted for by the ignored predictors in the residual sum of squares and hence inflate the residual mean square. These biases reflect as bias in prediction of estimates of the coefficients of retained variables and the response.  The general behavior of MSres as predictors (P) increases is as shown in the graph. As I mentioned earlier SSres always decreases as P increases, MSres initially decreases, then stabilizes, and eventually may increase. The eventual increase in MSres occurs when the reduction in SSres from adding a regressor to the model is not sufficient to compensate for the loss of one degree of freedom in the denominator of equation MSres = SSres/(n-p) i.e. adding a regressor to a p regressors model will cause new MSres (p+1) to be greater than MSres (p) if the decrease in SSres is less than MSres (p). This unstable behavior of MSres with the increase in predictors also governs the value of F-statistics and we can’t say where F-statistic increases or decreases for sure with increase in predictors in a model.

PS:  i am also not an expert in statistics tara maile regression analysis course liyeko thiye 

Last edited: 11-Dec-07 04:52 PM
Read Full Discussion Thread for this article