Major code organization and refactoring
Added logic for spider pausing and resuming