Overhaul of crawler assignment