CVE-2018-18314 (Other, published 2018-12-07T21:29Z)
Description: value: Perl before 5.26.3 has a buffer overflow via a crafted regular expression that triggers invalid write operations.
Severity: HIGH (CVSS v3 base score: 9.8)
Weakness: CWE-119 \u2014 Improper Restriction of Operations within the Bounds of a Memory Buffer
The product performs operations on a memory buffer, but it reads from or writes to a memory location outside the buffer's intended boundary. This may result in read or write operations on unexpected memory locations that could be linked to other variables, data structures, or internal program data.
Repository: https://github.com/Perl/perl5
Files changed:
pod/perldiag.pod
pod/perlrecharclass.pod
regcomp.c
t/lib/warnings/regcomp
t/re/reg_mesg.t
t/re/regex_sets.t
Commit message: fix #131649 - extended charclass can trigger assert

The extended charclass parser makes some assumptions during the
first pass which are only true on well structured input, and it
does not properly catch various errors. later on the code assumes
that things the first pass will let through are valid, when in
fact they should trigger errors.


Vulnerable code:
=item Syntax error in (?[...]) in regex m/%s/
C<< (?[ ]) >> is a regex-compile-time construct.  Any attempt to use
something which isn't knowable at the time the containing regular
                case '?':
                    if (RExC_parse[1] == '[') nest_depth++, RExC_parse++;
                    if (nest_depth--) break;
                    RExC_parse++;
                    if (*RExC_parse == ')') {
                    goto no_close;
      no_close:
        FAIL("Syntax error in (?[...])");
                    assert(UCHARAT(RExC_parse) == ')');
                    assert(UCHARAT(RExC_parse) == ')');
syntax error in (?[...]) in regex m/(?[[[:word]]])/ at - line 2.
syntax error in (?[...]) in regex m/(?[[[:digit: ])/ at - line 2.
syntax error in (?[...]) in regex m/(?[[:digit: ])/ at - line 2.
 '/(?[[[::]]])/' => "Syntax error in (?[...]) in regex m/(?[[[::]]])/",
 '/(?[[[:w:]]])/' => "Syntax error in (?[...]) in regex m/(?[[[:w:]]])/",
 '/(?[ \t ]/' => 'Syntax error in (?[...]) in regex m/(?[ \t ]/',
 '/(?[ [ \t ]/' => 'Syntax error in (?[...]) in regex m/(?[ [ \t ]/',
 '/(?[ \t ] ]/' => 'Syntax error in (?[...]) in regex m/(?[ \t ] ]/',
 '/(?[ [ ] ]/' => 'Syntax error in (?[...]) in regex m/(?[ [ ] ]/',
 '/(?[ \t + \e # This was supposed to be a comment ])/' => 'Syntax error in (?[...]) in regex m/(?[ \t + \e # This was supposed to be a comment ])/',
 '/ネ(?[[[:ネ]]])ネ/' => "Syntax error in (?[...]) in regex m/ネ(?[[[:ネ]]])ネ/",
 '/ネ(?[[[:ネ: ])ネ/' => "Syntax error in (?[...]) in regex m/ネ(?[[[:ネ: ])ネ/",
 '/ネ(?[[[::]]])ネ/' => "Syntax error in (?[...]) in regex m/ネ(?[[[::]]])ネ/",
 '/ネ(?[[[:ネ:]]])ネ/' => "Syntax error in (?[...]) in regex m/ネ(?[[[:ネ:]]])ネ/",
 '/ネ(?[ \t ]/' => 'Syntax error in (?[...]) in regex m/ネ(?[ \t ]/',
 '/(?[ \t + \e # ネ This was supposed to be a comment ])/' => 'Syntax error in (?[...]) in regex m/(?[ \t + \e # ネ This was supposed to be a comment ])/',
	like($@, qr/^Syntax error/, '/(?[(\c]) / should not panic');
	like($@, qr/^Syntax error/, '/(?[(\c]) ]\b/ should be a syntax error');
	like($@, qr/^Syntax error/, '/(?[\c[]](])/ should be a syntax error');

Fixed code:
=item Syntax error in (?[...]) in regex; marked by <-- HERE in m/%s/
=item Unexpected ']' with no following ')' in (?[... in regex; marked by <-- HERE in m/%s/

(F) While parsing an extended character class a ']' character was encountered
at a point in the definition where the only legal use of ']' is to close the
character class definition as part of a '])', you may have forgotten the close
paren, or otherwise confused the parser.

=item Expecting close paren for nested extended charclass in regex; marked by <-- HERE in m/%s/

(F) While parsing a nested extended character class like:

    (?[ ... (?flags:(?[ ... ])) ... ])
                             ^

we expected to see a close paren ')' (marked by ^) but did not.

=item Expecting close paren for wrapper for nested extended charclass in regex; marked by <-- HERE in m/%s/

(F) While parsing a nested extended character class like:

    (?[ ... (?flags:(?[ ... ])) ... ])
                              ^

we expected to see a close paren ')' (marked by ^) but did not.

Note that C<< (?[ ]) >> is a regex-compile-time construct.  Any attempt
to use something which isn't knowable at the time the containing regular
                case '(':
                    if (RExC_parse[1] == '?' && RExC_parse[2] == '[')
                        nest_depth++, RExC_parse+=2;
                    if (RExC_parse[1] == ')') {
                        RExC_parse++;
                        if (nest_depth--) break;
                    /* We output the messages even if warnings are off, because we'll fail
                     * the very next thing, and these give a likely diagnosis for that */
                    if (posix_warnings && av_tindex_skip_len_mg(posix_warnings) >= 0) {
                        output_or_return_posix_warnings(pRExC_state, posix_warnings, NULL);
                    }
                    RExC_parse++;
                    vFAIL("Unexpected ']' with no following ')' in (?[...");
        vFAIL("Syntax error in (?[...])");
                    if (UCHARAT(RExC_parse) != ')')
                        vFAIL("Expecting close paren for nested extended charclass");
                    if (UCHARAT(RExC_parse) != ')')
                        vFAIL("Expecting close paren for wrapper for nested extended charclass");
Unexpected ']' with no following ')' in (?[... in regex; marked by <-- HERE in m/(?[[[:word]] <-- HERE ])/ at - line 2.
syntax error in (?[...]) in regex; marked by <-- HERE in m/(?[[[:digit: ]) <-- HERE / at - line 2.
syntax error in (?[...]) in regex; marked by <-- HERE in m/(?[[:digit: ]) <-- HERE / at - line 2.

 '/(?[[[::]]])/' => "Unexpected ']' with no following ')' in (?[... {#} m/(?[[[::]]{#}])/",
 '/(?[[[:w:]]])/' => "Unexpected ']' with no following ')' in (?[... {#} m/(?[[[:w:]]{#}])/",
 '/(?[ \t ]/' => "Unexpected ']' with no following ')' in (?[... {#} m/(?[ \\t ]{#}/",
 '/(?[ [ \t ]/' => "Syntax error in (?[...]) {#} m/(?[ [ \\t ]{#}/",
 '/(?[ \t ] ]/' => "Unexpected ']' with no following ')' in (?[... {#} m/(?[ \\t ]{#} ]/",
 '/(?[ [ ] ]/' => "Syntax error in (?[...]) {#} m/(?[ [ ] ]{#}/",
 '/(?[ \t + \e # This was supposed to be a comment ])/' =>
    "Syntax error in (?[...]) {#} m/(?[ \\t + \\e # This was supposed to be a comment ]){#}/",
 '/ネ(?[[[:ネ]]])ネ/' => "Unexpected ']' with no following ')' in (?[... {#} m/ネ(?[[[:ネ]]{#}])ネ/",
 '/ネ(?[[[:ネ: ])ネ/' => "Syntax error in (?[...]) {#} m/ネ(?[[[:ネ: ])ネ{#}/",
 '/ネ(?[[[::]]])ネ/' => "Unexpected ']' with no following ')' in (?[... {#} m/ネ(?[[[::]]{#}])ネ/",
 '/ネ(?[[[:ネ:]]])ネ/' => "Unexpected ']' with no following ')' in (?[... {#} m/ネ(?[[[:ネ:]]{#}])ネ/",
 '/ネ(?[ \t ]/' => "Unexpected ']' with no following ')' in (?[... {#} m/ネ(?[ \\t ]{#}/",
 '/(?[ \t + \e # ネ This was supposed to be a comment ])/' =>
    "Syntax error in (?[...]) {#} m/(?[ \\t + \\e # ネ This was supposed to be a comment ]){#}/",
	like($@, qr/^Unexpected/, '/(?[(\c]) / should not panic');
	like($@, qr/^Unexpected/, '/(?[(\c]) ]\b/ should be a syntax error');
	like($@, qr/^Unexpected/, '/(?[\c[]](])/ should be a syntax error');