added support for buckets with more than 1000 files. bug fixes to support AWS API rate limits