* refactored recursive URL retrieval to be iterative to work on big datasets