Refactor RequestQueue to be a priority queue.

This ensures that all URLs of the same depth are crawled before
advancing to the next level.