Doing a recursive fetch will generate a lot of noise when the apache directory listings are used because links will be followed for sorting the file columns.