ENH speed up RBFSampler by ~10%

Applied the same to SkewedChi2Sampler, without measuring.

I was actually aiming for reduced memory usage, but unfortunately that
didn't matter much.