more site list based spider fixes to be more like gsa