Refactor url processing
list: argument for retrying failed urls