tinkerpop/gremlin — Issue #266: SailGraph(GraphSail(Neo4JGraph)) performance and classloader
Author: i000
Created: 2011-12-10 14:47:09+00:00
State: open

Hello,

This is more of a benchmark than a bug-report, but still it is an issue when dealing with large RDF data. I have found that the performance of a SailGraph(GraphSail(Neo4JGraph)) critically depends on the JVM memory settings and Gremlin's defaults are to stringent.

neo4j = new Neo4jGraph("data/bfg")
neo4j.setMaxBufferSize(100000)
graph = new SailGraph(new GraphSail(neo4j))

The graph is populated with ~1M nodes (via loadRDF) and I add edges programatically.

(v[X],v[Y]) = graph.LONG_GREMLIN_TRAVERSAL(U, V)
graph.addEdge(v[X], v[Y], "bfg:binds")

where U and V is parsed from a file. After an initial speed of 100-200 edges/s it drops to ~15 edges/s added in "bursts". This prompted me to increase the memory:

export JAVA_OPTIONS="-XX:MaxPermSize=2048M"

This fixes the performance drop for at least 1.5M edges.