Package: natex
Natural Language Expressions for PythonNatEx: Regular Expressions turbo-charged with notations for part-of-speech and dependency tree tagsIn a Nutshellfromnateximportnatexsentence=natex('Sloths eat steak in New York')# check if string begins with noun:sentence.match(r'@NOUN')# returns <natex.Match object; span=(0, 6), match='Sloths'># find first occurrence of an adposition followed by a proper nounsentence.search(r'@ADP <@PROPN>')# returns <natex.Match object; span=(17, 28), match='in New York'># find all occurrences of nouns or proper nounssentence.findall(r'@(NOUN|PROPN)')# returns ['Sloths', 'steak', 'New York']# find all occurrences of nouns or proper nouns starting with an s (irregardless of casing)sentence.findall(r's[^@]+@(NOUN|PROPN)',natex.I)# returns ['Sloths', 'steak']Goals & DesignThe goal of NatEx is quick and simple parsing of tokens using their literal representation, part-of-speech, and dependency tree tags.
Think of it as an extension ofregular expressionsfor natural language processing. The generated part-of-speech and dependency tree tags are provided bystanzaand merged into a string that can be searched through.Why notTregex,Semgrex, orTsurgeon?NatEx was designed primarily with simplicity in mind.
Libraries likeTregex,Semgrex, orTsurgeonmay be able to match more complex patterns, but they have a steep learning curve and the patterns are hard to read. Plus NatEx is written for Python. It wraps the built-inrepackage with an abstraction layer and thus provides almost the same performance as any normal regex.ExamplesYou can use it for simple tagging (NLU):fromnateximportnatexsentence=natex('book flights from Munich to Chicago')origin_location,destination_location=sentence.findall('<@PROPN>')# origin_location ='Munich', destination_location = 'Chicago'sentence=natex('I am being called Dan Borufka')firstname,lastname=sentence.findall('<@PROPN>')# firstname = 'Dan', lastname = 'Borufka'sentence=natex('I need to go to Italy')clause=sentence.search('<@ADP> <@PROPN>').match# clause = 'to Italy'destination=clause.split(' ')[1]Or for simple response template generation (NLG):fromnateximportnatexsentence=natex('Eat my shorts')# look for token with imperative formis_command=sentence.match(r'<!>')ifis_command:action_verb=sentence.search(r'<@VERB!>').lower()action_recipient=sentence.search(r'<#OBJ>')response=f'I will do my best to{action_verb}{action_recipient}!'# will contain 'I will do my best to eat shorts!'Even more (random) examples:fromnateximportnatexsentence=natex('Sloths eat steak in New York')# find first occurrence of character sequence "ea" in nouns onlysentence.search(r'ea@NOUN')# returns <natex.Match object; span=(11, 16), match='steak'># find first occurrence of character sequence "ea"sentence.search(r'ea')# returns <natex.Match object; span=(7, 9), match='ea'># find all occurrences of nouns or proper nouns starting with a lowercase ssentence.findall(r's[^@]+@(NOUN|PROPN)')# returns ['steak']sentence=natex('Ein Hund isst keinen Gurkensalat in New York.','de')# replace the nominal subject with the literal 'Affe'sentence.sub(r'#NSUBJ','Affe')# returns 'Ein Affe isst keinen Gurkensalat in New York.'Check outtest.pyfor some more examples!InstallationRun:pipinstallnatexBy default, NatEx only installs the English models for stanza.
Use the following command to download a model for another language:python-mnatexsetup<language_code>`e.g. for French use:python-mnatexsetupfr`Visithttps://github.com/secretsauceai/natex-pyfor a full list of supported language codes.UsageNatEx provides the same API as therepackage, but adds the following special characters:SymbolMeaningExample patternMeaning<token boundary (opening)<NewFind tokens starting with "New":either @ or #<:ADVFind tokens with e.g. universal POS "ADV" or dep. tree tag "ADVMOD"@part of speech tag@ADJFind tokens that are adjectives#dependency tree tag#OBJFind the objects of the sentence!imperative (mood)<!>Find any verbs that are in imperative form>token boundary (closing)York>Find all tokens ending in "York"If you combine features (e.g. seeking by part of speech and dependency tree simultaneously) make sure you provide them in the order of the table above.✔ This will work:natex('There goes a test sentence').findall(r'<@NOUN#OBJ>')✘ But this won't:natex('There goes a test sentence').findall(r'<#OBJ@NOUN>')Calling thenatex()function returns aNatExinstance. SeeAPIfor a more detailed description.
Just as there.Matchreturning methods provided by Python's built-inrepackage, NatEx' equivalents will return anatex.Matchobject containing aspanand amatchproperty referring to position and substring of the sentence respectively.ConfigurationYou can set theprocessing languageof NatEx using the second parameterlanguage_code(defaults to 'en').
It accepts a two-letter language-code, supportingall languages officially supported by stanza.sentence=natex('Das Faultier isst keinen Gurkensalat','de')When you run NatEx for the first time, it will check for the existence of the corresponding language models and download them if necessary. All subsequent calls tonatex()will exclude that step.APIThe API is derived from Python's built-inrepackage:NatEx.match(pattern, flags)Checks (from the beginning of the string) whether the sentence matches apatternand returns anatex.Matchobject orNoneotherwise..search(pattern, flags)Returns anatex.Matchobject describing the first substring matchingpattern..findall(pattern, flags)Returns all found strings matchingpatternas a list..split(pattern, flags)Splits the sentence by all occurrences of the foundpatternand returns a list of strings..sub(pattern, replacement, flags)Replaces all occurrences of the foundpatternbyreplacementand returns the changed string.TestingYou can use pytest in your terminal (simply type inpytest) to run the unit tests shipped with this package.
Install it by runningpip install pytestin your terminal.