document args to regexec_flags and API

Document in the API, and clarify in the source code, what the arguments
to Perl_regexec_flags are.

NB: this info is based on code inspection, not any real knowledge on my
part.
PL_sawampersand: use 3 bit flags rather than bool

Set a separate flag for each of $`, $& and $'.
It still works fine in boolean context.

This will allow us to have more refined control over what parts
of a match string to copy (we currently copy the whole string).
regexec_flags(): simplify length calculation

The code to calculate the length of the string to copy was

    PL_regeol - startpos + (stringarg - strbeg);

This is a hangover from the original (perl 3) regexp implementation
that under //i, copied and folded the original buffer: so startpos might
not equal stringarg. These days it always is (except under a match failure
with (*COMMIT), and the code we're interested is only executed on success).

So simplify to just PL_regeol - strbeg.
Separate handling of ${^PREMATCH} from $` etc

Currently the handling of getting the value, length etc of ${^PREMATCH}
etc is identical to that of $` etc.

Handle them separately, by adding RX_BUFF_IDX_CARET_PREMATCH etc
constants to the existing RX_BUFF_IDX_PREMATCH set.

This allows, when retrieving them, to always return undef if the current
match didn't use //p. Previously the result depended on stuff such
as whether the (non-//p) pattern included captures or not.

The documentation for ${^PREMATCH} etc states that it's only guaranteed to
return a defined value when the last pattern was //p.

As well as making things more consistent, this is a necessary
prerequisite for the following commit, which may not always copy the
whole string during a non-//p match.
Don't copy all of the match string buffer

When a pattern matches, and that pattern contains captures (or $`, $&, $'
or /p are present), a copy is made of the whole original string, so
that $1 et al continue to hold the correct value even if the original
string is subsequently modified. This can have severe performance
penalties; for example, this code causes a 1Mb buffer to be allocated,
copied and freed a million times:

    $&;
    $x = 'x' x 1_000_000;
    1 while $x =~ /(.)/g;

This commit changes this so that, where possible, only the needed
substring of the original string is copied: in the above case, only a
1-byte buffer is copied each time. Also, it now reuses or reallocs the
buffer, rather than freeing and mallocing each time.

Now that PL_sawampersand is a 3-bit flag indicating separately whether
$`, $& and $' have been seen, they each contribute only their own
individual penalty; which ones have been seen will limit the extent to
which we can avoid copying the whole buffer.

Note that the above code *without* the $& is not currently slow, but only
because the copying is artificially disabled to avoid the performance hit.
The next but one commit will remove that hack, meaning that it will still
be fast, but will now be correct in the presence of a modified original
string.

We achieve this by by adding suboffset and subcoffset fields to the
existing subbeg and sublen fields of a regex, to indicate how many bytes
and characters have been skipped from the logical start of the string till
the physical start of the buffer. To avoid copying stuff at the end, we
just reduce sublen. For example, in this:

    "abcdefgh" =~ /(c)d/

subbeg points to a malloced buffer containing "c\0"; sublen == 1,
and suboffset == 2 (as does subcoffset).

while if $& has been seen,

subbeg points to a malloced buffer containing "cd\0"; sublen == 2,
and suboffset == 2.

If in addition $' has been seen, then

subbeg points to a malloced buffer containing "cdefgh\0"; sublen == 6,
and suboffset == 2.

The regex engine won't do this by default; there are two new flag bits,
REXEC_COPY_SKIP_PRE and REXEC_COPY_SKIP_POST, which in conjunction with
REXEC_COPY_STR, request that the engine skip the start or end of the
buffer (it will still copy in the presence of the relevant $`, $&, $',
/p).

Only pp_match has been enhanced to use these extra flags; substitution
can't easily benefit, since the usual action of s///g is to copy the
whole string first time round, then perform subsequent matching iterations
against the copy, without further copying. So you still need to copy most
of the buffer.
rationalise t/re/pat_psycho.t

Do some cleanup of this file, without changing its functionality.

Once upon a time, the psycho tests were scattered throughout a single
pat.t file, before being moved into their own file. Now that they're all
in a single file, make the $PERL_SKIP_PSYCHO_TEST test a single "skip_all"
test at the beginning of the file, rather than testing it separately in
each code block.

Also, make some of the test descriptions more useful, and add a bit of
debugging output.
stop $foo =~ /(bar)/g skipping copy

Normally in the presence of captures, a successful regex execution
makes a copy of the matched string, so that $1 et al give the right
value even if the original string is changed; i.e.

    $foo =~ /(123)/g;
    $foo = "bar";
    is("$1", "123");

Until now that test would fail, because perl used to skip the copy for
the scalar /(...)/g case (but not the C<$&; //g> case). This was to
avoid a huge slowdown in code like the following:

    $x = 'x' x 1_000_000;
    1 while $x =~ /(.)/g;

which would otherwise end up copying a 1Mb string a million times.

Now that (with the last commit but one) we copy only the required
substring of the original string (a 1-byte substring in the above
example), we can remove this fast-but-incorrect hack.
tidy up patten match copying code

(no functional changes).

1. Remove some dead code from pp_split; it's protected by an assert
that it could never be called.

2. Simplify the flags settings for the call to CALLREGEXEC() in
pp_substcont: on subsequent matches we always set REXEC_NOT_FIRST,
which forces the regex engine not to copy anyway, so passing the
REXEC_COPY_STR is pointless, as is the conditional code to set it.

3. (whitespace change): split a conditional expression over 2 lines
for easier reading.
m// and s///; don't copy TEMP/AMAGIC strings

Currently pp_match and pp_subst make a copy of the match string if it's
SvTEMP(), and in the case of pp_match, also if it's SvAMAGIC().

This is no longer necessary, as the code will always copy the string
anyway if its actually needed after the match, i.e. if it detects the
presence of $1, $& or //p etc. Until a few commits ago, this wasn't the
case for pp_match: it would sometimes skip copying even in the presence of
$1 et al for efficiency reasons. Now that that's fixed, we can remove the
SvTEMP() and SvAMAGIC() tests.

As to why pp_subst did the SvTEMP test, I don't know: but removing it
didn't make any tests fail!
fix a bug in handling $+[0] and unicode

The code to decide what substring of a pattern target to copy for the
sake of $1, $& etc, would, in the absence of $&, only copy the minimum
range needed to cover $1,$2,...., which might be a shorter range than
what $& covers. This is fine most of the time, but, when calculating
$+[0] on a unicode string, it needs a copy of the whole part of the string
covered by $&, since it needs to convert the byte offest into a char
offset.
So to fix this, always copy as a minimum, the $& range.
I suppose we could be more clever about this: detect the presence
of @+ in the code, only do it for UTF8 etc; but this is simple
and non-fragile.
[MERGE] only copy bits of regex match string

When making a copy of the string being matched against (so that $1, $&
et al continue to show the correct value even if the original string is
subsequently modified), only copy that substring of the original string
needed for the capture variables, rather than copying the whole string.

This is a big win for code like

    $&;
    $_ = 'x' x 1_000_000;
    1 while /(.)/;

Also, when pessimizing if the code contains $`, $& or $', record
the presence of each variable separately, so that the determination of the
substring range is based on each variable separately. So performance-wise,

   $&; /x/

is now roughly equivalent to

   /(x)/

whereas previously it was like

   /^(.*)(x)(.*)$/

and

   $&; $'; /x/

is now roughly equivalent to

   /(x)(.*)$/

etc.

Finally, this code (when not in the presence of $& etc)

    $_ = 'x' x 1_000_000;
    1 while /(.)/;

used to skip the buffer copy for performance reasons, but suffered from $1
etc changing if the original string changed. That's now been fixed too.