added in paging to limit results, and some general refactoring