do not re-read the same directory when spidering
boegel test added