Package: hmrb
Hammurabi [hmrb] 🏺Upholds the law for sequences.1. InstallationTo begin, simply install the package from PyPI:$pipinstallhmrb2. DocumentationDocumentation is available athttps://hmrb.readthedocs.io.
Instructions to build and run locally:$pipinstall-rdoc_requirements.txt
$pipinstall-e.
$makedocs
$makehtml3. DefinitionsHammurabi works as a rule engine to parse input using a defined set of rules.
It uses a simple and readable syntax to define complex rules to handle phrase matching.The engine takes as input any type of sequences of units with associated attributes.
Our usecase currently is to handle language annotation, but we expect it to work
equally well on a variety of complex sequence tasks (time-series, logging).The attributes do not have to be consistent across all units or between the
units and the grammar. The lack of an attribute is simply considered as a
non-match.Features:Attribute level rule definitions using key-values pairsEfficient matching of sequence using hash tables with no limit on lengthSupport for nested boolean expressions and wildcard operators similar to regular expressionsVariables can be side-loaded and reused throughout different rule setsUser-defined rule-level callback functions triggered by a matchLabels to tag and retrieve matched sequence segments3.1 Writing RulesRules are defined in a custom syntax. The syntax was defined
with the aim to keep it simple to read, but expressive at the same time.The basic components areLawandVar. BothLawandVardeclare a sequence of attributes.
However, while aLawcan be matched on its own, aVardefines a sequence that is likely to be reused (a.k.a macros) withinLawsor otherVars. Since aVaris never matched on its own, it requires a name and only exists as part of a rule body.The example below shows a fictional case of capturing strings such as"head is hurting"or"head hurts".
Note that the variableis_hurtingcannot matchis hurting.Var is_hurting:
(
    optional (lemma: "be")
    (lemma: "hurt")
)

Law:
    - package: "headache"
    - callback: "mark_headache"
    - junk_attribute: "some string"
(
    (lemma: "head", pos: "NOUN")
    $is_hurting
)3.2 Input formatHammurabi requires a sequence of attribute dictionaries as input.
It will attempt to find matching rules in the given input.
The most widely-used input format is a simple JSON list of dictionaries:[{"orth":"My","lemma":"my","pos":"PRON"},{"orth":"head","lemma":"head","pos":"NOUN"},{"orth":"hurts","lemma":"hurt","pos":"VERB"}]3.3 Callbacks, labels and dataWhen a rule matches an input, the following information is returned as a
"match": the original input, a slice representing the span it was triggered on
and all the data (labels, callback function and attributes) based on
the matched rule. There are two ways to act upon these matches.
You can use delegate the execution of the callback function tohammurabior you can do the execution yourself. The former is done by passing the input
to the__call__method, which executes callback functions right after
the matches are returned. However, this has a slight drawback, which is that
your callback functions need to adhere to a specific signature to allow them
to be called correctly from insidehammurabi.# callback function called from inside hammurabidefmark_headache(input_,slice_,data):print(f'I am acting on span "{input_[slice_]}" with data "{data}".')The callback functions are passed down as a mapping between their string alias
used in the rule grammar, i.e. how do you refer to it in thecallbackattribute of the law that was matched.callbacks={'mark_headache':mark_headache}4. Usage4.1  Worked-out example with callbacksThe rule engine is initialized through aCoreinstance. We can pass various optional
objects to the constructor ofCore(callbacks, sets) that we intend to later use in our rules.TheCore.loadmethod adds rules to the engine.
It is possible to load multiple rule files sequentially.TheCorelibrary usage pattern allows the user to either get the
matches and act on them in a different place through the use of thematchmethod, or to pass a callback mapping and allowhammurabito execute the
callbacks through the use of the__call__method.grammar="""Var is_hurting:(optional (lemma: "be")(lemma: "hurt"))Law:- package: "headache"- callback: "mark_headache"- junk_attribute: "some string"((lemma: "head", pos: "NOUN")$is_hurting)"""input_=[{"orth":"My","lemma":"my","pos":"PRON"},{"orth":"head","lemma":"head","pos":"NOUN"},{"orth":"hurts","lemma":"hurt","pos":"VERB"},]# Library use casefromhmrb.coreimportCorespans=[(start,input_[start:])forstartinrange(len(input_))]hmb_ext=Core()hmb_ext.load(grammar)# external executionforspan,datainhmb_ext._match(spans):print("External execution!!!")slice_=slice(span[0],span[1])callbacks[data[0]["callback"]](input_,slice_,data)# External execution!!!# I am acting on span "head hurts" with data# "{#      'package': 'headache',#      'callback': 'mark_headache',#      'junk_attribute': 'some string'# }"# internal executionhmb_int=Core(callbacks={"mark_headache":mark_headache})hmb_int.load(grammar)hmb_int(input_)#  I am acting on span "head hurts" with data#  "{#       'package': 'headache',#       'callback': 'mark_headache',#       'junk_attribute': 'pointless strings I am passing down because I can'#  }"You can find this worked out example underexamples/readme.py.4.2 spaCy component example (NLP)The spaCy component classSpacyCoreextends the internal execution shown
above to allow the use ofhammurabiin spaCy natural language processing
pipelines. Optionally a function (jsonify) can be passed into the SpacyCore
to convert theTokenobjects to JSON.importspacyfromhmrb.coreimportSpacyCore# This will be used to turn a span (subsequence) of a spaCy document object# into a list of dictionaries input representation.defjsonify(span):jsn=[]fortokeninspan:jsn.append({'orth':token.orth_,'lemma':token.lemma_,'pos':token.pos_,'tag':token.tag_})returnjsnhmb=SpacyCore(callbacks={'mark_headache':mark_headache},map_doc=jsonify,sort_length=True)hmb.load(grammar)nlp=spacy.load('en_core_web_sm')nlp.add_pipe(hmb,last=True)nlp('My head hurts')#  I am acting on span "head hurts" with data#  "{#       'package': 'headache',#       'callback': 'mark_headache',#       'junk_attribute': 'pointless strings I am passing down because I can'#  }"5. Tests & debuggingTo run tests use (this inclused setting the correctHASH_SEED):$maketestsTo display additional information for debugging purposes useDEBUG=1environment variable.$DEBUG=1pythonexample.py6. MaintainersKristian BodaSasho SavkovMaria LehlMade with