Ensure the spider's worker is shut down if an exception occurs