Package: rdflib-hdt
A Store back-end forrdflibto allow for reading and querying HDT documents.Online DocumentationRequirementsPythonversion 3.6.4 or higherpipgcc/clangwithc++11 supportPython Development headers..You should have thePython.hheader available on your system.For example, for Python 3.6, install thepython3.6-devpackage on Debian/Ubuntu systems.InstallationInstallation usingpipenvor avirtualenvisstrongly advised!PyPi installation (recommended)# you can install using pippipinstallrdflib-hdt# or you can use pipenvpipenvinstallrdflib-hdtManual installationRequirement:pipenvgitclonehttps://github.com/Callidon/pyHDTcdpyHDT/./install.shGetting startedYou can use therdflib-hdtlibrary in two modes: as an rdflib Graph or as a raw HDT document.Graph usage (recommended)fromrdflibimportGraphfromrdflib_hdtimportHDTStorefromrdflib.namespaceimportFOAF# Load an HDT file. Missing indexes are generated automatically# You can provide the index file by putting them in the same directory than the HDT file.store=HDTGraph("test.hdt")# Display some metadata about the HDT document itselfprint(f"Number of RDF triples:{len(store)}")print(f"Number of subjects:{store.nb_subjects}")print(f"Number of predicates:{store.nb_predicates}")print(f"Number of objects:{store.nb_objects}")print(f"Number of shared subject-object:{store.nb_shared}")Using the RDFlib API, you can alsoexecute SPARQL queriesover an HDT document.
If you do so, we recommend that you first call theoptimize_sparqlfunction, which optimize
the RDFlib SPARQL query engine in the context of HDT documents.fromrdflibimportGraphfromrdflib_hdtimportHDTStore,optimize_sparql# Calling this function optimizes the RDFlib SPARQL engine for HDT documentsoptimize_sparql()graph=Graph(store=HDTStore("test.hdt"))# You can execute SPARQL queries using the regular RDFlib APIqres=graph.query("""
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name ?friend WHERE {
   ?a foaf:knows ?b.
   ?a foaf:name ?name.
   ?b foaf:name ?friend.
}""")forrowinqres:print(f"{row.name}knows{row.friend}")HDT Document usagefromrdflib_hdtimportHDTDocument# Load an HDT file. Missing indexes are generated automatically.# You can provide the index file by putting them in the same directory than the HDT file.document=HDTDocument("test.hdt")# Display some metadata about the HDT document itselfprint(f"Number of RDF triples:{document.total_triples}")print(f"Number of subjects:{document.nb_subjects}")print(f"Number of predicates:{document.nb_predicates}")print(f"Number of objects:{document.nb_objects}")print(f"Number of shared subject-object:{document.nb_shared}")# Fetch all triples that matches { ?s foaf:name ?o }# Use None to indicates variablestriples,cardinality=document.search_triples((None,FOAF("name"),None))print(f"Cardinality of (?s foaf:name ?o):{cardinality}")fors,p,ointriples:print(triple)# The search also support limit and offsettriples,cardinality=document.search_triples((None,FOAF("name"),None),limit=10,offset=100)# etc ...An HDT document also provides support for evaluating joins over a set of triples patterns.fromrdflib_hdtimportHDTDocumentfromrdflibimportVariablefromrdflib.namespaceimportFOAF,RDFdocument=HDTDocument("test.hdt")# find the names of two entities that know each othertp_a=(Variable("a"),FOAF("knows"),Variable("b"))tp_b=(Variable("a"),FOAF("name"),Variable("name"))tp_c=(Variable("b"),FOAF("name"),Variable("friend"))query=set([tp_a,tp_b,tp_c])iterator=document.search_join(query)print(f"Estimated join cardinality:{len(iterator)}")# Join results are produced as ResultRow, like in the RDFlib SPARQL APIforrowiniterator:print(f"{row.name}knows{row.friend}")Handling non UTF-8 strings in pythonIf the HDT document has been encoded with a non UTF-8 encoding the previous code won’t work correctly and will result in aUnicodeDecodeError.
More details on how to convert string to str from C++ to PythonhereTo handle this, we doubled the API of the HDT document by adding:search_triples_bytes(...)return an iterator of triples as(py::bytes,py::bytes,py::bytes)search_join_bytes(...)return an iterator of sets of solutions mapping aspy::set(py::bytes,py::bytes)convert_tripleid_bytes(...)return a triple as:(py::bytes,py::bytes,py::bytes)convert_id_bytes(...)return apy::bytesParameters and documentation are the same as the standard versionfromrdflib_hdtimportHDTDocumentdocument=HDTDocument("test.hdt")it=document.search_triple_bytes("","","")fors,p,oinit:print(s,p,o)# print b'...', b'...', b'...'# now decode it, or handle any errortry:s,p,o=s.decode('UTF-8'),p.decode('UTF-8'),o.decode('UTF-8')exceptUnicodeDecodeErroraserr:# try another other codecs, ignore error, etcpass