codeforamerica/citygram — pull request #124 by invisiblefunnel (2014-10-08 21:47:22+00:00)
Title: Short circuit pagination if a page with no new records is returned

Citygram will queue jobs for up to ten sequential pages if the `Next-Page` header is given. It may not always be necessary to pull all those pages. One indicator is when a page in the sequence yields no new events. In this case, bail out of the pagination chain. If this becomes limiting, let's reevaluate. For now, I'm comfortable that this is the right behavior. Please note, there is an important assumption baked into pagination: that the endpoint orders events to return the latest first.