Bugfix for crawler poop-out on follow 404