Package: eldar
Boolean text search using EldarGetting StartedThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.InstallingYou can install the method by typing:pip install eldarBasic usagefromeldarimportQuery# build listdocuments=["Gandalf is a fictional character in Tolkien's The Lord of the Rings","Frodo is the main character in The Lord of the Rings","Ian McKellen interpreted Gandalf in Peter Jackson's movies","Elijah Wood was cast as Frodo Baggins in Jackson's adaptation","The Lord of the Rings is an epic fantasy novel by J. R. R. Tolkien"]eldar=Query('("gandalf" OR "frodo") AND NOT ("movie" OR "adaptation")')# use `filter` to get a list of matches:print(eldar.filter(documents))# >>> ["Gandalf is a fictional character in Tolkien's The Lord of the Rings",#     'Frodo is the main character in The Lord of the Rings']# call to see if the text matches the query:print(eldar(documents[0]))# >>> True# by default, words must match. Thus, "movie" != "movies":print(eldar(documents[2]))# >>> TrueYou can also use it to mask Pandas DataFrames:fromeldarimportQueryimportpandasaspd# build dataframedf=pd.DataFrame(["Gandalf is a fictional character in Tolkien's The Lord of the Rings","Frodo is the main character in The Lord of the Rings","Ian McKellen interpreted Gandalf in Peter Jackson's movies","Elijah Wood was cast as Frodo Baggins in Jackson's adaptation","The Lord of the Rings is an epic fantasy novel by J. R. R. Tolkien"],columns=['content'])# build query objecteldar=Query('("gandalf" OR "frodo") AND NOT ("movie" OR "adaptation")')# eldar's call returns True if the text matches the query.# You can filter a dataframe using pandas mask syntax:df=df[df.content.apply(eldar)]print(df)ParametersThere are three parameters that you can adjust in the query builder.
By default:Query(...,ignore_case=True,ignore_accent=True,match_word=True)Let the query bequery = '"movie"':Ifignore_caseis True, the documents "Movie" and "movie" will be matched. If False, only "movie" will be matched.Ifignore_accentis True, the documents "mövie" will be matched.Ifmatch_wordis True, the document will be tokenized and the query terms will have to match exactly. If set to False, the documents "movies" and "movie" will be matched. Setting this option to True may slow down the query.WildcardsQueries also support*as wildcard character. Wildcard matches any number (including none) of alphanumeric characters.fromeldarimportQuery# sample document and query with multiple wildcards:document="Gandalf is a fictional character in Tolkien's The Lord of the Rings"eldar=Query('"g*dal*"')# call to see if the text matches the query:print(eldar(document))# >>> TrueBuilding an index for faster queriesSearching in a large corpus using the Query object is slow, as each document has to be checked.
For (much) faster queries, create anIndexobject, and build it using a list of documents.fromeldarimportIndexfromeldar.trieimportTriedocuments=["Gandalf is a fictional character in Tolkien's The Lord of the Rings","Frodo is the main character in The Lord of the Rings","Ian McKellen interpreted Gandalf in Peter Jackson's movies","Elijah Wood was cast as Frodo Baggins in Jackson's adaptation","The Lord of the Rings is an epic fantasy novel by J. R. R. Tolkien","Frodo Baggins is a hobbit"]index=Index(ignore_case=True,ignore_accent=True)index.build(documents)# must only be done once# persist and retrieve index from diskindex.save("index.p")# but documents are copied to diskindex=Index.load("index.p")print(index.search('"frodo b*" AND NOT hobbit'))# support wildcardsprint(index.count('"frodo b*" AND NOT hobbit'))# shows only the count# to only return document ids, set `return_ids` to True:print(index.search('"frodo b*" AND NOT hobbit',return_ids=True))It works like a usual search engine does: by keeping a dictionary that maps each word to its document ids. The boolean query is turned into an operation tree, where document ids are joined or intersected in order to return the desired matches.LicenseThis package is MIT licensed.