Package: rebulk
ReBulkReBulk is a python library that performs advanced searches in strings
that would be hard to implement usingre
moduleorString
methodsonly.It includes some features likePatterns,Match,Rulethat allows
developers to build a custom and complex string matcher using a readable
and extendable API.This project is hosted on GitHub:https://github.com/Toilal/rebulkInstall$pipinstallrebulkUsageRegular expression, string and function based patterns are declared in aRebulkobject. It use a fluent API to chainstring,regex, andfunctionalmethods to define various patterns types.>>>fromrebulkimportRebulk>>>bulk=Rebulk().string('brown').regex(r'qu\w+').functional(lambdas:(20,25))WhenRebulkobject is fully configured, you can callmatchesmethod
with an input string to retrieve allMatchobjects found by registered
pattern.>>>bulk.matches("The quick brown fox jumps over the lazy dog")[<brown:(10,15)>,<quick:(4,9)>,<jumps:(20,25)>]If multipleMatchobjects are found at the same position, only the
longer one is kept.>>>bulk=Rebulk().string('lakers').string('la')>>>bulk.matches("the lakers are from la")[<lakers:(4,10)>,<la:(20,22)>]String PatternsString patterns are based onstr.findmethod to find matches, but returns all matches in the string.ignore_casecan be enabled to ignore case.>>>Rebulk().string('la').matches("lalalilala")[<la:(0,2)>,<la:(2,4)>,<la:(6,8)>,<la:(8,10)>]>>>Rebulk().string('la').matches("LalAlilAla")[<la:(8,10)>]>>>Rebulk().string('la',ignore_case=True).matches("LalAlilAla")[<La:(0,2)>,<lA:(2,4)>,<lA:(6,8)>,<la:(8,10)>]You can define several patterns with a singlestringmethod call.>>>Rebulk().string('Winter','coming').matches("Winter is coming...")[<Winter:(0,6)>,<coming:(10,16)>]Regular Expression PatternsRegular Expression patterns are based on a compiled regular expression.re.finditermethod is used to find matches.Ifregex moduleis available, it
can be used by rebulk instead of defaultre
module. Enable it withREBULK_REGEX_ENABLED=1environment variable.>>>Rebulk().regex(r'l\w').matches("lolita")[<lo:(0,2)>,<li:(2,4)>]You can define several patterns with a singleregexmethod call.>>>Rebulk().regex(r'Wint\wr',r'com\w{3}').matches("Winter is coming...")[<Winter:(0,6)>,<coming:(10,16)>]All keyword arguments fromre.compileare
supported.>>>importre# import required for flags constant>>>Rebulk().regex('L[A-Z]KERS',flags=re.IGNORECASE)\....matches("The LaKeRs are from La")[<LaKeRs:(4,10)>]>>>Rebulk().regex('L[A-Z]','L[A-Z]KERS',flags=re.IGNORECASE)\....matches("The LaKeRs are from La")[<La:(20,22)>,<LaKeRs:(4,10)>]>>>Rebulk().regex(('L[A-Z]',re.IGNORECASE),('L[a-z]KeRs'))\....matches("The LaKeRs are from La")[<La:(20,22)>,<LaKeRs:(4,10)>]Ifregex moduleis available, it
automatically supports repeated captures.>>># If regex module is available, repeated_captures is True by default.>>>matches=Rebulk().regex(r'(\d+)(?:-(\d+))+').matches("01-02-03-04")>>>matches[0].children# doctest:+SKIP[<01:(0,2)>,<02:(3,5)>,<03:(6,8)>,<04:(9,11)>]>>># If regex module is not available, or if repeated_captures is forced to False.>>>matches=Rebulk().regex(r'(\d+)(?:-(\d+))+',repeated_captures=False)\....matches("01-02-03-04")>>>matches[0].children[<01:(0,2)+initiator=01-02-03-04>,<04:(9,11)+initiator=01-02-03-04>]abbreviationsDefined as a list of 2-tuple, each tuple is an abbreviation. It
simply replacetuple[0]withtuple[1]in the expression.>>> Rebulk().regex(r'Custom-separators',
abbreviations=[("-", r"[W_]+")])...
.matches("Custom_separators using-abbreviations")
[<Custom_separators:(0, 17)>]Functional PatternsFunctional Patterns are based on the evaluation of a function.The function should have the same parameters asRebulk.matchesmethod,
that is the input string, and must return at least start index and end
index of theMatchobject.>>>deffunc(string):...index=string.find('?')...ifindex>-1:...return0,index-11>>>Rebulk().functional(func).matches("Why do simple ? Forget about it ...")[<Why:(0,3)>]You can also return a dict of keywords arguments forMatchobject.You can define several patterns with a singlefunctionalmethod call,
and function used can return multiple matches.Chain PatternsChain Patterns are ordered composition of string, functional and regex
patterns. Repeater can be set to define repetition on chain part.>>>r=Rebulk().regex_defaults(flags=re.IGNORECASE)\....defaults(children=True,formatter={'episode':int,'version':int})\....chain()\....regex(r'e(?P<episode>\d{1,4})').repeater(1)\....regex(r'v(?P<version>\d+)').repeater('?')\....regex(r'[ex-](?P<episode>\d{1,4})').repeater('*')\....close()# .repeater(1) could be omitted as it's the default behavior>>>r.matches("This is E14v2-15-16-17").to_dict()# converts matches to dictMatchesDict([('episode',[14,15,16,17]),('version',2)])Patterns parametersAll patterns have options that can be given as keyword arguments.validatorFunction to validateMatchvalue given by the pattern. Can also be
adict, to usevalidatorwith pattern named with key.>>>defcheck_leap_year(match):...returnint(match.value)in[1980,1984,1988]>>>matches=Rebulk().regex(r'\d{4}',validator=check_leap_year)\....matches("In year 1982 ...")>>>len(matches)0>>>matches=Rebulk().regex(r'\d{4}',validator=check_leap_year)\....matches("In year 1984 ...")>>>len(matches)1Some base validator functions are available inrebulk.validatorsmodule. Most of those functions have to be configured usingfunctools.partialto map them to function accepting a singlematchargument.formatterFunction to convertMatchvalue given by the pattern. Can also be
adict, to useformatterwith matches named with key.>>>defyear_formatter(value):...returnint(value)>>>matches=Rebulk().regex(r'\d{4}',formatter=year_formatter)\....matches("In year 1982 ...")>>>isinstance(matches[0].value,int)Truepre_match_processor/post_match_processorFunction to mutagen or invalidate a match generated by a pattern.Function has a single parameter which is the Match object. If
function returns False, it will be considered as an invalid match.
If function returns a match instance, it will replace the original
match with this instance in the process.post_processorFunction to change the default output of the pattern. Function
parameters are Matches list and Pattern object.nameThe name of the pattern. It is automatically passed toMatchobjects generated by this pattern.tagsA list of string that qualifies this pattern.valueOverride value property for generatedMatchobjects. Can also be adict, to usevaluewith pattern named with key.validate_allBy default, validator is called for returnedMatchobjects only.
Enable this option to validate them all, parent and children
included.format_allBy default, formatter is called for returnedMatchvalues only.
Enable this option to format them all, parent and children included.disabledAfunction(context)to disable the pattern if returningTrue.childrenIfTrue, all childrenMatchobjects will be retrieved instead of
a single parentMatchobject.privateIfTrue,Matchobjects generated from this pattern are available
internally only. They will be removed at the end ofRebulk.matchesmethod call.private_parentForce parent matches to be returned and flag them as private.private_childrenForce children matches to be returned and flag them as private.private_namesMatches names that will be declared as privateignore_namesMatches names that will be ignored from the pattern output, after
validation.markerIftrue,Matchobjects generated from this pattern will be
markers matches instead of standard matches. They won't be included
inMatchessequence, but will be available inMatches.markerssequence (seeMarkerssection).MatchAMatchobject is the result created by a registered pattern.It has avalueproperty defined, and position indices are available
throughstart,endandspanproperties.In some case, it contains childrenMatchobjects inchildrenproperty, and each childMatchobject reference its parent inparentproperty. Also, anameproperty can be defined for the match.If groups are defined in a Regular Expression pattern, each group match
will be converted to a singleMatchobject. If a group has a name
defined ((?P<name>group)), it is set asnameproperty in a childMatchobject. The whole regexp match (re.group(0)) will be converted
to the mainMatchobject, and all subgroups (1, 2, ... n) will be
converted tochildrenmatches of the mainMatchobject.>>>matches=Rebulk()\....regex(r"One, (?P<one>\w+), Two, (?P<two>\w+), Three, (?P<three>\w+)")\....matches("Zero, 0, One, 1, Two, 2, Three, 3, Four, 4")>>>matches[<One,1,Two,2,Three,3:(9,33)>]>>>forchildinmatches[0].children:...'%s=%s'%(child.name,child.value)'one = 1''two = 2''three = 3'It's possible to retrieve only children by usingchildrenparameters.
You can also customize the way structure is generated withevery,private_parentandprivate_childrenparameters.>>>matches=Rebulk()\....regex(r"One, (?P<one>\w+), Two, (?P<two>\w+), Three, (?P<three>\w+)",children=True)\....matches("Zero, 0, One, 1, Two, 2, Three, 3, Four, 4")>>>matches[<1:(14,15)+name=one+initiator=One,1,Two,2,Three,3>,<2:(22,23)+name=two+initiator=One,1,Two,2,Three,3>,<3:(32,33)+name=three+initiator=One,1,Two,2,Three,3>]Match object has the following properties that can be given to Pattern
objectsformatterFunction to convertMatchvalue given by the pattern. Can also be
adict, to useformatterwith matches named with key.>>>defyear_formatter(value):...returnint(value)>>>matches=Rebulk().regex(r'\d{4}',formatter=year_formatter)\....matches("In year 1982 ...")>>>isinstance(matches[0].value,int)Trueformat_allBy default, formatter is called for returnedMatchvalues only.
Enable this option to format them all, parent and children included.conflict_solverAfunction(match, conflicting_match)used to solve conflict.
Returned object will be removed from matches byConflictSolverdefault rule. If__default__string is returned, it will fallback
to default behavior keeping longer match.MatchesAMatchesobject holds the result ofRebulk.matchesmethod call.
It's a sequence ofMatchobjects and it behaves like a list.All methods accepts apredicatefunction to filterMatchobjects
using a callable, and anindexint to retrieve a single element from
default returned matches.It has the following additional methods and properties on it.starting(index, predicate=None, index=None)Retrieves a list ofMatchobjects that starts at given index.ending(index, predicate=None, index=None)Retrieves a list ofMatchobjects that ends at given index.previous(match, predicate=None, index=None)Retrieves a list ofMatchobjects that are previous and nearest to
match.next(match, predicate=None, index=None)Retrieves a list ofMatchobjects that are next and nearest to
match.tagged(tag, predicate=None, index=None)Retrieves a list ofMatchobjects that have the given tag defined.named(name, predicate=None, index=None)Retrieves a list ofMatchobjects that have the given name.range(start=0, end=None, predicate=None, index=None)Retrieves a list ofMatchobjects for given range, sorted from
start to end.holes(start=0, end=None, formatter=None, ignore=None, predicate=None, index=None)Retrieves a list ofholeMatchobjects for given range. A hole
match is created for each range where no match is available.conflicting(match, predicate=None, index=None)Retrieves a list ofMatchobjects that conflicts with given match.chain_before(self, position, seps, start=0, predicate=None, index=None):Retrieves a list of chained matches, before position, matching
predicate and separated by characters from seps only.chain_after(self, position, seps, end=None, predicate=None, index=None):Retrieves a list of chained matches, after position, matching
predicate and separated by characters from seps only.at_match(match, predicate=None, index=None)Retrieves a list ofMatchobjects at the same position as match.at_span(span, predicate=None, index=None)Retrieves a list ofMatchobjects from given (start, end) tuple.at_index(pos, predicate=None, index=None)Retrieves a list ofMatchobjects from given position.namesRetrieves a sequence of allMatch.nameproperties.tagsRetrieves a sequence of allMatch.tagsproperties.to_dict(details=False, first_value=False, enforce_list=False)Convert to an ordered dict, withMatch.nameas key andMatch.valueas value.It's a subclass ofOrderedDict,
that contains amatchesproperty which is a dict withMatch.nameas key and list ofMatchobjects as value.Iffirst_valueisTrueand distinct values are found for the
same name, value will be wrapped to a list. IfFalse, first value
only will be kept and values lists can be retrieved withvalues_listwhich is a dict withMatch.nameas key and list ofMatch.valueas value.ifenforce_listisTrue, all values will be wrapped to a list,
even if a single value is found.Ifdetailsis True,Match.valueobjects are replaced with
completeMatchobject.markersA customMatchessequences specialized formarkersmatches (see
below)MarkersIf you have defined some patterns withmarkersproperty, thenMatches.markerspoints to a specialMatchessequence that contains
onlymarkersmatches. This sequence supports all methods fromMatches.Markers matches are not intended to be used in final result, but can be
used to implement aRule.RulesRules are a convenient and readable way to implement advanced
conditional logic involving severalMatchobjects. When a rule is
triggered, it can perform an action onMatchesobject, like filtering
out, adding additional tags or renaming.Rules are implemented by extending the abstractRuleclass. They are
registered usingRebulk.rulemethod by giving either aRuleinstance, aRuleclass or a module containingRule classesonly.For a rule to be triggered,Rule.whenmethod must returnTrue, or a
non empty list ofMatchobjects, or any other truthy object. When
triggered,Rule.thenmethod is called to perform the action withwhen_responseparameter defined as the response ofRule.whencall.Instead of implementingRule.thenmethod, you can defineconsequenceclass property with a Consequence classe or instance, likeRemoveMatch,RenameMatchorAppendMatch. You can also use a list
of consequence when required :when_responsemust then be iterable,
and elements of this iterable will be given to each consequence in the
same order.When many rules are registered, it can be useful to setpriorityclass
variable to define a priority integer between all rule executions
(higher priorities will be executed first). You can also definedependencyto declare another Rule class as dependency for the current
rule, meaning that it will be executed before.For all rules with the samepriorityvalue,whenis called before,
andthenis called after all.>>>fromrebulkimportRule,RemoveMatch>>>classFirstOnlyRule(Rule):...consequence=RemoveMatch......defwhen(self,matches,context):...grabbed=matches.named("grabbed",0)...ifgrabbedandmatches.previous(grabbed):...returngrabbed>>>rebulk=Rebulk()>>>rebulk.regex("This match(.*?)grabbed",name="grabbed")<...Rebulkobject...>>>>rebulk.regex("if it's(.*?)first match",private=True)<...Rebulkobjectat...>>>>rebulk.rules(FirstOnlyRule)<...Rebulkobjectat...>>>>rebulk.matches("This match is grabbed only if it's the first match")[<Thismatchisgrabbed:(0,21)+name=grabbed>]>>>rebulk.matches("if it's NOT the first match, This match is NOT grabbed")[]Changelogv3.2.0 (2023-02-18)Featuredependencies:Add python 3.11 support and drop python 3.6 support (e4cb0d8)FixRemove pytest-runner from setup_requires (4483d17)v3.1.0 (2021-11-04)Featuredefaults:Add overrides support (#25) (f79e5ea)python:Add python 3.10 support, drop python 3.5 support (a5e6eb7)v3.0.1 (2020-12-25)Fixpackage:Fix broken packageNo such file or directory: 'CHANGELOG.md'(#24) (33895ff)Documentationreadme:Add semantic release badge (78baca0)readme:Fix title (d5d4db5)v3.0.0 (2020-12-23)Featureregex:Replace REGEX_DISABLED environment variable with REBULK_REGEX_ENABLED (d5a8cad)Add python 3.8/3.9 support, drop python 2.7/3.4 support (048a15f)Breakingregex module is now disabled by default, even if it's available in the python interpreter. You have to set REBULK_REGEX_ENABLED=1 in your environment to enable it, as this module may cause some issues.  (d5a8cad)Python 2.7 and 3.4 support have been dropped  (048a15f)