Draft: draft-bormann-jsonpath-iregexp

Commentary: Implemented the changes as suggested in Issue 
 The purpose of this issue is to start discussion about how to turn this into a completed draft to send to the IESG . Perhaps the most important issue is how much of the explanatory and motivational material to leave in . There is a case to be made for taking all of it out , so you get a nice short RFC that just says what I - Regexp is . So here 's my initial straw - person proposal on what should be removed and changed : Remove the " issues " para in Section 3 . Section 5.2 , 5.3 , remove the trailing note about improving performance . This is an implementation artifact , no ? Section 6 . Replace with a single short paragraph somewhat as follows : " While regular expressions originally were intended to describe a formal language , i.e. , to provide a Boolean matching function , they have turned into parsing functions for many applications . With this accretion of features , parsing regexp libraries have become susceptible to bugs and performance degradation , in particular those that can be exploited in Denial of Service ( DoS ) attacks in the case where an attacker controls the regex submitted for processing . I - Regexp is designed to offer interoperability , and to be less vulnerable to such attacks , with the trade - off that its only function is to offer a boolean response as to whether a character sequence is matched by a regex . [ If we really feel that other parts of this discussion add value , they could migrate into an appendix . ] 6.1 - cut this down , retaining the first para of the first bullet point , the first sentence of the second bullet point , and that 's all . The Unicode discussion can migrate into     by
The present specification defines an interoperable regular expression
flavor, I-Regexp.
This specification describes an interoperable regular expression
flavor, I-Regexp.
I-Regexp supports the entire repertoire of Unicode characters.
The grammatical rules in this document are to be interpreted as ABNF,
as described in RFC5234.
This is essentially XSD regexp without character class subtraction
and multi-character escapes.
This is essentially XSD regexp without character class subtraction
and multi-character escapes such as "\s", "\S", and "\w".
Note that where a regexp literal is required, this needs to enclose
the actual regexp in "/".
The performance of an ECMAScript matcher can be increased by turning
parenthesized regexps (last choice in production "atom") into
"(?:...)" constructions.
Note that where a regexp literal is required, the actual regexp needs
to be enclosed in "/".
Envelope the result in "\A" and "\z".
Again, the performance can be increased by turning parenthesized
regexps (production "atom") into "(?:...)" constructions.
Data modeling formats (YANG, CDDL) as well as query languages
(jsonpath) often need a regular expression (regexp) sublanguage.
There are many dialects of regular expressions in use in platforms,
programming languages, and data modeling formats.
While regular expressions originally were intended to describe a
formal language, i.e., to provide a Boolean matching function, they
have turned into parsing functions for many applications, with
capture groups, greedy/lazy/possessive variants, etc.  Language
features such as backreferences allow specifying languages that
actually are context-free (Chomsky type 2) instead of the regular
languages (Chomsky type 3) that regular expressions are named for.
YANG (RFC7950) and CDDL (RFC8610) have adopted the regexp language
from W3C Schema XSD-2.  XSD regexp is a pure matching language, i.e.,
XSD regexps can be used to match a string against them and yield a
simple true or false result.  XSD regexps are not as widely
implemented as programming language regexp dialects such as those of
Perl, Python, Ruby, Go RE2, or JavaScript (ECMAScript) ECMA-262.  The
latter are often in a state of continuous development; in the best
case (ECMAScript) there is a complete specification which however is
highly complex (Section 21.2 of ECMA-262 comprises 62 pages) and
evolves on a yearly timeline, with significant additions.  Regexp
dialects such as PCRE PCRE2 have evolved to cover a common set of
functions available in parsing regexp dialects, offered in a widely
available library.
With continuing accretion of complex features, parsing regexp
libraries have become susceptible to bugs and performance
degradation, in particular those that can be exploited in Denial of
Service (DoS) attacks.  The library RE2 that is compatible with Go
language regexps strives to be immune to DoS attacks, making it
attractive to applications such as query languages where an attacker
could control the input.  The problem remains that other bugs in such
libraries can lead to exploitable vulnerabilities; at the time of
writing, the Common Vulnerabilities and Exposures (CVE) system has
131 entries that mention the word "regex" REGEX-CVE (not all, but
many of which are such bugs, with 23 matches for arbitrary code
execution).
Implementations of YANG and CDDL often struggle with providing true
XSD regexps; some instead cheat by providing one of the parsing
regexp varieties, sometimes without even advertising this fact.
A matching regexp that does not use the more complex XSD features
(subsetting) can usually be converted into a parsing regexp of many
dialects by simply surrounding it with anchors of that dialect (e.g.,
"^" or "\A" and "$" or "\z").  If the original matching regexps
exceed the envelope of compatibility between dialects, this can lead
to interoperability problems, or, worse, security vulnerabilities.
Also, features of the target dialect such as capture groups may be
triggered inadvertently, reducing performance.
Character class subtraction.  This is a very useful feature in
many specifications, but it is unfortunately mostly absent from
parsing regexp dialects.
Discussion: This absence can often be addressed by translating
character class subtraction into positive character classes
(possibly requiring significant expansion) and/or inserting
negative lookahead assertions (which are not universally supported
by regexp libraries, most notably not by RE2 RE2).  This
specification therefore opts for leaving out character class
subtraction.
Unicode.  While there is no doubt that a regexp flavor meant to
last needs to be Unicode enabled, there are a number of aspects of
this that need discussion.  Not all regexp implementations that
one might want to map I-Regexps into will support accesses to
Unicode tables that enable executing on constructs such as
"\p{IsCoptic}", for mapping into such implementations, translation
needs to be provided.  Fortunately, the "\p"/"\P" feature in
general is now quite widely available.
Enclose the regexp in "\A" and "\z".
While regular expressions originally were intended to describe a
formal language to support a Boolean matching function, they have
been enhanced with parsing functions that support the extraction and
replacement of arbitrary portions of the matched text.  With this
accretion of features, parsing regexp libraries have become more
susceptible to bugs and surprising performance degradations which can
be exploited in Denial of Service attacks by an attacker who controls
the regexp submitted for processing.  I-Regexp is designed to offer
interoperability, and to be less vulnerable to such attacks, with the
trade-off that its only function is to offer a boolean response as to
whether a character sequence is matched by a regexp.
Character class subtraction.  This is a very useful feature in
many specifications, but it is unfortunately mostly absent from
parsing regexp dialects.  Thus, it is omitted from I-Regexp.
Multi-character escapes.  "\d", "\w", "\s" and their uppercase
complement classes exhibit a large amount of variation between
regexp flavors.  Thus, they are omitted from I-Regexp.
Not all regexp implementations support accesses to Unicode tables
that enable executing on constructs such as "\p{IsCoptic}",
although the "\p"/"\P" feature in general is now quite widely
available.  While in principle it's possible to translate these
into codepoint-range matches, this also requires access to those
tables.  Thus, regexp libraries in severely constrained
environments may not be able to support I-Regexp conformance.
As discussed in background, more complex regexp libraries are likely
to contain exploitable bugs leading to crashes and remote code
execution.  There is also the problem that such libraries often have
hard to predict performance characteristics, leading to attack
vectors that overload an implementation by matching against an
expensive attacker-controlled regexp.
As discussed in background, more complex regexp libraries may contain
exploitable bugs leading to crashes and remote code execution.  There
is also the problem that such libraries often have hard-to-predict
performance characteristics, leading to attacks that overload an
implementation by matching against an expensive attacker-controlled
regexp.