Peter,
I have also done this. I used the same shim stock I use for the pivot, just cut a bit smaller then the spacer.
Tim makes a good point, now, looseness in general, quite probably will have a negative effect on accuracy. It will also hide a poorly fit barrel. (Which does nothing for accuracy) Test the fit or looseness of a barrel without the forend attached.
CW