versae/neo4j-rest-client — Issue #49: Batch inefficiency - requesting 5 nodes leads to 6 HTTP requests
Author: mhluongo
Created: 2011-12-19 04:13:53+00:00
State: open

I thought the batch support would be a great boon to performance, but it only helps for writes and indexing in the db. Here's an example:

nodes = []
with conn.transaction():
for i in range(5):
nodes.append(conn.node[i])

... while this looks good, and does in fact pull all the nodes in a batch, right after the transaction ends `client.Node`'s constructor is called 5 times, one for each new node. Unfortunately, the constructor calls the database again, which leads to more requests than if we hadn't batched in the first place. The requests look something like:

POST http://localhost:7474/db/data/batch
[[('to', u'/node/0'), ('method', 'GET'), ('id', 0)],
[('to', u'/node/1'), ('method', 'GET'), ('id', 1)],
[('to', u'/node/2'), ('method', 'GET'), ('id', 2)],
[('to', u'/node/3'), ('method', 'GET'), ('id', 3)],
[('to', u'/node/4'), ('method', 'GET'), ('id', 4)]]
GET http://localhost:7474/db/data/node/0
GET http://localhost:7474/db/data/node/1
GET http://localhost:7474/db/data/node/2
GET http://localhost:7474/db/data/node/3
GET http://localhost:7474/db/data/node/4

(where the list below the POST is the attached json body)

We've subclassed `Node` to avoid this particular behavior, but this sort of eager access to the database can be found throughout the client. Any sort of improvement would be really helpful.

To anyone else with this problem- I don't have time to submit a patch, but I can provide our workarounds if you need them.