Package: tantivy
tantivy-pyPython bindings forTantivythe full-text search engine library written in Rust.InstallationThe bindings can be installed using from pypi using pip:pip install tantivyIf no binary wheel is present for your operating system the bindings will be
build from source, this means that Rust needs to be installed before building
can succeed.Note that the bindings are usingPyO3, which
only supports python3.DevelopmentFor compiling Python module:# create virtual envpython-mvenv.venvsource.venv/bin/activate# install maturin, the build tool for PyO3pipinstallmaturin# compile and install python module in venvmaturindevelopSetting up a development environment can be done in a virtual environment usingnoxor using local packages using the providedMakefile.For thenoxsetup install the virtual environment and build the bindings using:python3 -m pip install nox
noxFor theMakefilebased setup run:makeRunning the tests is done using:make testUsageThe Python bindings have a similar API to Tantivy. To create a index first a schema
needs to be built. After that documents can be added to the index and a reader
can be created to search the index.Building an index and populating itimporttantivy# Declaring our schema.schema_builder=tantivy.SchemaBuilder()schema_builder.add_text_field("title",stored=True)schema_builder.add_text_field("body",stored=True)schema_builder.add_integer_field("doc_id",stored=True)schema=schema_builder.build()# Creating our index (in memory)index=tantivy.Index(schema)To have a persistent index, use the path
parameter to store the index on the disk, e.g:index=tantivy.Index(schema,path=os.getcwd()+'/index')By default, tantivy  offers the following tokenizers
which can be used in tantivy-py:defaultdefaultis the tokenizer that will be used if you do not
assign a specific tokenizer to your text field.
It will chop your text on punctuation and whitespaces,
removes tokens that are longer than 40 chars, and lowercase your text.rawDoes not actual tokenizer your text. It keeps it entirely unprocessed.
It can be useful to index uuids, or urls for instance.en_stemIn addition to whatdefaultdoes, theen_stemtokenizer also
apply stemming to your tokens. Stemming consists in trimming words to
remove their inflection. This tokenizer is slower than the default one,
but is recommended to improve recall.to use the above tokenizers, simply provide them as a parameter toadd_text_field. e.g.schema_builder.add_text_field("body",stored=True,tokenizer_name='en_stem')Adding one document.writer=index.writer()writer.add_document(tantivy.Document(doc_id=1,title=["The Old Man and the Sea"],body=["""He was an old man who fished alone in a skiff in the Gulf Stream and he had gone eighty-four days now without taking a fish."""],))# ... and committingwriter.commit()Building and Executing QueriesFirst you need to get a searcher for the index# Reload the index to ensure it points to the last commit.index.reload()searcher=index.searcher()Then you need to get a valid query object by parsing your query on the index.query=index.parse_query("fish days",["title","body"])(best_score,best_doc_address)=searcher.search(query,3).hits[0]best_doc=searcher.doc(best_doc_address)assertbest_doc["title"]==["The Old Man and the Sea"]print(best_doc)Valid Query Formatstantivy-py supports the query language used in tantivy.
Some basic query Formats.AND and OR conjunctions.query=index.parse_query('(Old AND Man) OR Stream',["title","body"])(best_score,best_doc_address)=searcher.search(query,3).hits[0]best_doc=searcher.doc(best_doc_address)+(includes) and -(excludes) operators.query=index.parse_query('+Old +Man chef -fished',["title","body"])(best_score,best_doc_address)=searcher.search(query,3).hits[0]best_doc=searcher.doc(best_doc_address)Note: in a query like above, a word with no +/- acts like an OR.phrase search.query=index.parse_query('"eighty-four days"',["title","body"])(best_score,best_doc_address)=searcher.search(query,3).hits[0]best_doc=searcher.doc(best_doc_address)integer searchquery=index.parse_query('"eighty-four days"',["doc_id"])(best_score,best_doc_address)=searcher.search(query,3).hits[0]best_doc=searcher.doc(best_doc_address)Note: for integer search, the integer field should be indexed.For more possible query formats and possible query options, seeTantivy Query Parser Docs.