Package: prosecco
proseccoDescriptionNLP engine with text extraction capabilities that can be easily extended to desired needs.Can be used to build chat bots, question answer machines (seeexample/qa.py), text converters.Extract words or even whole sentences in ordered manner.Get position of found text.UseConditionclass and mark data using regex or string comparasion.Extend each part of it in easy manner. ( seeexample/custom_condition_class.py).InstallpipinstallproseccoUsageBasicexample/basic.pyfromproseccoimportProsecco,Condition,EnglishWordNormalizer# Read wikipedia https://en.wikipedia.org/wiki/Superherowithopen("superhero.txt")asf:text=f.read()# 1. Create conditions with hero namesconditions=[Condition(lemma_type="hero|dc",compare=["batman","superman","wonder woman"],lower=True),Condition(lemma_type="hero|marvel",normalizer=EnglishWordNormalizer(),compare=["spiderman","iron man","black panther"],lower=True)]# 2. Create proseccop=Prosecco(conditions=conditions)# 3. Let's drink and print outputp.drink(text,progress=True)lemmas=set(p.get_lemmas(type="hero"))print(" ".join(map(str,lemmas)))OutputBatman[hero|dc][start:1089] Wonder Woman[hero|dc][start:2100] Iron Man[hero|marvel][start:2184] Superman[hero|dc][start:2070] Spider-Man[hero|marvel][start:2080] Black Panther[hero|marvel][start:17690]Advancedexample/advanced.pyfromproseccoimport*text="""Chrząszcz brzmi w trzcinie w Szczebrzeszynie.Ząb zupa zębowa, dąb zupa dębowa.Gdzie Rzym, gdzie Krym. W Pacanowie kozy kują.Tak, jeśli mam szczęśliwy być, to w Gdańsku muszę żyć!"""# 1. Create condition with city namescities=["szczebrzeszyn","pacanow","gdansk","rzym","krym"]animals=["koz","chrzaszcz"]# 2. Normalizer to remove polish specific charsetn=CharsetNormalizer(Charset.PL_EN)# 3. Stemmer to remove suffixs=SuffixStemmer(language="pl")# 4. Conditions for city and animalcity_condition=Condition(lemma_type="city",compare=cities,normalizer=n,stemmer=s,lower=True)animal_condition=Condition(lemma_type="animal",compare=animals,normalizer=n,stemmer=s,lower=True)conditions=[city_condition,animal_condition]# 5. Create tokenizer for polish charsettokenizer=LanguageTokenizer(Charset.PL)# 6. Get list of tokenstokens=tokenizer.tokenize(text)# 7. Create visitor with conditions provided in step 1visitor=Visitor(conditions=conditions)# 8. Parse tokens based on visitor conditionslexer=Lexer(tokens=tokens,visitor=visitor)# 9. Get list of lemmaslemmas=lexer.lex()# 10. filter found cities and print outputfound=filter(lambdal:l.type=="city",lemmas)print(" ".join(map(str,found)))# 11. filter found anumals and print outputfound=list(filter(lambdal:l.type=="animal",lemmas))print(" ".join(map(str,found)))# 12. print exact words from textforlinlist(found):print(text[l.start:l.start+len(l.sentence)])OutputSzczebrzeszynie[city][start:29]Rzym[city][start:86]Krym[city][start:98]Pacanowie[city][start:106]Gdańsku[city][start:163]Chrząszcz[animal][start:0]kozy[animal][start:116]Chrząszcz
kozyQA ( question answer machine )example/qa.pyfromdatetimeimportdatetimefromproseccoimportProsecco,Condition,EnglishWordNormalizer,SuffixStemmermessages="""Whats the time ?How long boil egg?100 miles to km30,3 celsius to farenheit"""# create conditionsquestion=("what","whats","how")measure=("celsius","farenheit","mile","km","kilometer","time","long")cooking=("boil","cook","fry")food=("egg",)conditions=[Condition(lemma_type="question",compare=question,normalizer=EnglishWordNormalizer(),lower=True),Condition(lemma_type="measure",compare=measure,normalizer=EnglishWordNormalizer(),stemmer=SuffixStemmer(language="en"),lower=True),Condition(lemma_type="cooking",compare=cooking,normalizer=EnglishWordNormalizer(),lower=True),Condition(lemma_type="food",compare=food,normalizer=EnglishWordNormalizer(),stemmer=SuffixStemmer(language="en"),lower=True),Condition(lemma_type="number",compare=r"\d+([\.\,]\d+)?",regex=True,until_character=" "),]defprinter(data):print("Robot : ",data)# time conditiondefresolve_time(p):printer(datetime.now())# cooking conditiondefresolve_cooking(p):ifcheck_condition(p.get_lemmas("cooking|food"),["boil","egg"]):printer("""Hard for 9-15 minutes.Soft for 6-8 minutes.""")returnTruedefresolve_measure(p):measures=p.get_lemmas("measure")fr=measures[0]to=measures[1]numbers=p.get_lemmas("number")iflen(numbers)==0:printer("No number for conversion provided")returnTruevalue=float(numbers[0].sentence.replace(",","."))iffr.condition=="mile"andto.condition=="km":printer(value/0.62137119)returnTrueeliffr.condition=="km"andto.condition=="mile":printer(value*0.62137119)returnTrueeliffr.condition=="celsius"andto.condition=="farenheit":print(9/5*value+32)returnTrueeliffr.condition=="farenheit"andto.condition=="celsius":print((value-32)*5/9)returnTruereturnFalsedefcheck_condition(lemmas,conditions):forlinlemmas:forcinconditions:ifl.condition==c:conditions.remove(c)returnlen(conditions)==0defresolve(p,m):iflen(p.get_lemmas("question"))>0:ifcheck_condition(p.get_lemmas("measure"),["time"]):resolve_time(p)returnTrueeliflen(p.get_lemmas("cooking"))>0:returnresolve_cooking(p)eliflen(p.get_lemmas("measure"))>0:returnresolve_measure(p)returnFalseforminmessages.split('\n'):print("Question : ",m)p=Prosecco(conditions=conditions)p.drink(m)ifnotresolve(p,m):print("Unsupported resolver : ",p.lemmas)OutputQuestion:Whatsthetime?
Robot:2019-08-1320:38:06.948720
Question:Howlongboilegg?
Robot:Hardfor9-15minutes.
Softfor6-8minutes.
Question:100milestokm
Robot:160.93440057946685
Question:30,3celsiustofarenheit86.53999999999999