Bootstrap for regression

[v***@yahoo.fr]

Hi experts,

I am working with a (very) small sample size (n=22 european countries). I do a multiple linear regression with 2 predictors. Then I try to use the bootstrap for least squares.

Of course the bootstrap would be more efficient if the sample size was larger, but I guess that the bootstrapping techniques are also useful when the sample size is small. In that case, when the sample size is small, using bootstrap (permutation or randomization tests), we can "reach" or approximate the exact P-value for the t student test of the regression's coefficients, no ?
And the prediction intervals of the regression are going to be more precise, more reliable using bootstrap, no ?
Is it right to say that the bootstrap will give "better" "more precise" "more reliable" results ?
Or the bootstrap will just give results that are at least as good as the normal approximation ?


Thanks for your help.
Best Regards, looking forward to reading You.

[Rich Ulrich]

Post by v***@yahoo.fr
Hi experts,
I am working with a (very) small sample size (n=22 european countries). I do a multiple linear regression with 2 predictors. Then I try to use the bootstrap for least squares.
Of course the bootstrap would be more efficient if the sample size was larger,

- "efficient" is not a word that I would use here.

I mention this because your post gets into the topics discussed in
statistical estimation theory, which uses a number of terms in
precise ways. An "efficient" estimator is one that makes good use
of the available information, regardless of N.

Post by v***@yahoo.fr
but I guess that the bootstrapping techniques are also useful when the sample size is small. In that case, when the sample size is small,
using bootstrap (permutation or randomization tests),

Terminology? For this regression, I think that the simple
permutation solution would permute the choices of Outcome
to be matched to a set of predictors. Bootstrapping, by contrast,
entails a randomized draw-with-replacement, done many times.

Post by v***@yahoo.fr
we can "reach" or approximate the exact P-value for the t student test of the regression's coefficients, no ?

Permutation in the fashion I described could give *one version*
of an exact test on the overall F for two variables. I don't
imagine the same test applying to the tests on coefficients, though.
I've never seen someone argue for tests on a pair of coefficients.

Post by v***@yahoo.fr
And the prediction intervals of the regression are going to be more precise, more reliable using bootstrap, no ?

Absolutely not. Bootstrap is usually discussed under the heading
of "robust statistics". The idea of opting for robustness is to gain
protection against violation of the assumptions; the trade-off
accepted is that the results of the original regression will be
"more efficient" and thus be more reliable and have smaller
intervals for the instances where the assumptions are met well
enough.

Post by v***@yahoo.fr
Is it right to say that the bootstrap will give "better" "more precise" "more reliable" results ?
Or the bootstrap will just give results that are at least as good as the normal approximation ?

As described above, No guarantee. Might be better or worse,
depending on violations. Probably not much different for simple
regression. My impression is that bootstrapping is mostly advised
for situations with more complications.