andreasronge/neo4j — Issue #130: Better performance for pagination
Author: dnagir
Created: 2012-01-15 03:40:39+00:00
State: open
Labels: improvement

Reasoning
---

The current pagination looks pretty inefficient. It loops through all the items to get to the page every time, be it 50 items or 5000000.

So we'll start kill the application going further and further to the end.

We also need (not sure if that's available already):

- `skip(N)` - a way to efficiently skip the first N items
- `limit(N)` - efficiently limit the result set to N items.
- `sort` (probably is already available) - efficient way to sort.

The pagination and/or "infinite scrolling" is easily achieved by combining those methods:

`items.sort(:name).skip(25000).limit(10)` which is supposed to be efficient as :)

Then we could just add a a convenience method `paginate` where it makes sense. The implementation will look something like this:

```ruby
def paginate(options={})
page = options[:page] || 1
per_page = options[:per_page] || 20
self.skip((per_page-1) * page).limit(per_page)
end
```

Ins
--------

I tried to get rid of all `wp_query` methods to see where those are used. And got the failing specs in:

- `Neo4j::HasN::Mapping`
- `Neo4j::Traversal::Traverser`
- `Model` itself (presumable based on `.all`)
- `Neo4j::Index::LuceneQuery`
- `Neo4j::Rails::Relationships::NodesDSL`
- `Neo4j::Rails::Relationships::RelsDSL`

We'll need to implement the common interface for all of those (something like current WillPaginate) that would leverage the underlying DB pagination support.

A good start can be figuring out how [Creating a paged traverser](http://docs.neo4j.org/chunked/milestone/rest-api-traverse.html#rest-api-creating-a-paged-traverser) is implemented and take that approach.