fixed some bugs in the spiders. tested small data