Note on database layouts for SPARQL datastores

Abstract: The SPARQL query language for RDF provides a standardized way to access Semantic Web data. This report summarizes some lessons learnt while implementing a SPARQL datastore on top of ModelRDB, the database backend of the Jena Semantic Web Framework, and puts forward recommendations for the database layout of a future, dedicated SPARQL datastore. This report summarizes some lessons the author has learnt while implementing the sparql2sql triple store [2]. sparql2sql is based on the database backend of the Jena Semantic Web Framework [1], which is also known as ModelRDB [6]. I argue that the ModelRDB database layout, while being a good persistent backend for the Jena API, is not a good match for a SPARQL datastore. My recommendations for a better schema include: 1. using a normalized representation of RDF triples, 2. using an index on the GraphID column, 3. using a separate index on the Predicate column, 4. using the same node encoding for S, P, O and GraphID, 5. using separate tables for the default graph and the named graphs of an RDF dataset, 6. storing model metadata in a form accessible to SQL. The recommendations put forward here are largely consistent with the findings presented in [3], which, at the time of writing, is the most comprehensive work on database layouts for SPARQL stores. 1 Normalized and denormalized layout Contrary to common database design wisdom, ModelRDB uses a denormalized schema. URIs and literals are stored directly in the statement table. This decision is justified in [6]: ModelRDB has been optimized for find queries, that is, for finding all triples matching a simple triple pattern where subject, predicate and object may be wildcards. The denormalized schema avoids the need for database joins when performing find queries. This improves performance. There are, however, several reasons why a normalized schema works better for more complex SPARQL queries. SPARQL requires joins anyway. SPARQL queries typically involve several or many triple patterns. This means joins cannot be avoided. Furthermore, the columns used in join conditions represent RDF nodes. In a denormalized table, the joins are made over big string columns. In a normalized table, the joins are made over key columns, which are typically integers. Joins over integers are faster because less data has to be retrieved from disk. SPARQL queries have higher selectivity. Find queries are not very discriminate. A find query will often return a sizable chunk of the datastore. Much filtering will be done in application code. This means lots of node data has to be accessed per find query. SPARQL queries are much more accurate in pinpointing the bits of data required by the client. Most of the processing is done by matching triples against each other; the actual node representations don’t have to be accessed unless the node is in the results. The number of accesses into the nodes table of a normalized schema might actually be quite low. Memory savings: A normalized schema occupies less space on disk because each node representation is stored only once, even if the node occurs in many triples. For a store that mostly has to support find queries, a denormalized schema works better. For a store that mostly has to support SPARQL, a normalized schema should work better. The question is where the balance tips. Some informal tests suggest that a denormalized schema might be faster for queries involving one and two triple patterns, and might be slower for three or more triple patterns.