weavejester/clj-aws-s3 — pull request #29 by sw1nn (2013-05-02 06:01:16+00:00)
Title: Add support to handle paged results from list-objects & list-versions.

This adds some support for handling paged results.

Background: when you make a call to `list-objects` or `list-versions` and the number of items of data returned exceeds the `max-keys` (for list-objects) or `max-results` (for list-versions) options, then the `truncated?` flag is set to true. You can then get the next page by passing various identifiers from the current results into a request for more results.

This adds `paged-results-seq` which maps those paging into calls to `lazy-cat` and can be used to build the other calls

To handle the common cases where you just want a seq of objects or versions there are wrappers `list-objects-seq` and 'list-versions-seq`. By using these you don't get access to the other meta-data in the result object, but I don't believe you need this most of the time (and it you do you can use `paged-results-seq` directly.

The AWS api has `listNextBatchOf*` methods, which could be used to the same effect, but the lazy-seq approach seems a better clojure way to do it.

What do you think?