Dataset index caching and pagination.

This fixes #655 in issue tracker on github.
This fixes #642 in issue tracker on github.