Update executor based crawler to accept the executor service

Now we can shutdown the executor pool.