Stop constant inlining from countermanding ‘use subs’

Ever since

commit f7461760003db2ce68155c97ea6c1658e96fcd27
Author: Zefram <zefram@fysh.org>
Date:   Sun Nov 8 15:03:45 2009 +0100

    Bareword sub lookups
    ...

this has failed:

$ perl5.10 -le 'use subs "abs";  sub abs() {44}; print abs + abs'
88
$ perl5.12 -le 'use subs "abs";  sub abs() {44}; print abs + abs'
44

A GV holding a single constant is a candidate for downgrading after
it uhas been used.  The GV gets downgraded after the first ‘abs’ is
inlined.  In the process, the CV-imported flag, which is stored in the
GV, not the CV, is lost, preventing &abs from overriding the built-in
function on the second mention.

There is a special flag for RVs, namely SVprv_PCS_IMPORTED,
which indicates that, when expanded to GVs, they should have the
GVf_IMPORTED_CV flag set.  But gv_try_downgrade wasn‘t setting
that flag.
[perl #118237] Fix coreamp.t’s rand test

when rand returns something really small that does not
begin with 0, such as 2.90736361456823e-05.
Update Pod-Usage to CPAN version 1.63

  [DELTA]

1.63 (marekr)
- CPAN#85477: Module version not updated in last release
  ...fixed
- CPAN#85478: typo fixes
  ...corrected
lexsub.t: To-do tests for citing lex subs after errors

This currently causes assertion failures on debugging builds.  On
non-debugging builds (untested), it probably crashes:

my sub a { foo ref } # foo must not exist
a();
Name lexical constants

$ ./perl -Ilib -Mfeature=:all -e 'my sub a(){44} a()'
The lexical_subs feature is experimental at -e line 1.
Assertion failed: (hek), function Perl_ck_subr, file op.c, line 10558.
Abort trap: 6

The experimental warning is expected.  The assertion failure is not.

When a call checker is invoked, the name of the subroutine is passed
to it.  op.c:ck_subr gets the name from the CV’s cv (CvGV) or, in the
case of lexical subs, from its name hek (CvNAME_HEK).  If neither
exists, ck_subr cannot cope.

Lexical subs never have a GV pointer.  Lexical constants were acci-
dentally having neither a GV pointer nor a hek.  They should have a
hek, like other lexical subs.
[perl #116735] Honour lexical prototypes when no parens are used

As Peter Martini noted in ticket #116735, lexical subs produce dif-
ferent op trees for ‘foo 1’ and ‘foo(1)’.  foo(1) produces an rv2cv
op with a padcv kid.  The unparenthetical version produces just
a padcv op.

And the difference in op trees caused lexical sub calls to honour
prototypes only in the presence of parentheses, because rv2cv_op_cv
(which searches for the cv in order to check its prototype) was
expecting rv2cv+padcv.

Not realising there was a discrepancy between the two forms, and
noticing that foo() produces *two* newCVREF ops, in commit 279d09bf893
I made newCVREF return just a padcv op for lexical subs.  At the time
I couldn’t figure out why there were two rv2cv ops, and punted on
researching it.

This is how it works for package subs:

When a sub call is compiled, if there are parentheses, an implicit '&'
is fed to the parser.  The token that follows is a WORD token with a
constant op attached to it, containing the name of the subroutine.
When the parser sees '&', it calls newCVREF on the const op to create
an rv2cv op.

For sub calls without parentheses, the token passed to the parser is
already an rv2cv op.

The resulting op tree is the same either way.

For lexical subs, I had the lexer emitting an rv2cv op in both paths,
which was why we got the double rv2cv when newCVREF was returning an
rv2cv for lexical subs.

The real solution is to call newCVREF in the lexer only when there
are no parentheses, since in that case the lexer is not going to call
newCVREF itself.  That avoids a redundant newCVREF call.  Hence, we
can have newCVREF always return an rv2cv op.

The result is that ‘foo(1)’ and ‘foo 1’ produce identical op trees for
a lexical sub.

One more thing needed to change:  The lexer was not looking at the
lexical prototype CV but simply the stub to be autovivified, so it
couldn’t see the parameter prototype attached to the CV (the stub
doesn’t have one).

The lexer needs to see the parameter prototype too, in order to deter-
mine precedence.

The logic for digging through pads to find the CV has been extracted
out of rv2cv_op_cv into a separate (non-API!) routine.
Fix crashes after syntax errors in lexical subs

Peter Martini fixed this in commit 89e006ae4e39db for our subs.
(Thank you, BTW, if you are reading this.)

The warning is expected; the assertion failure is not:

$ ./miniperl -Ilib -Mfeature=:all -e 'state sub a { is ref } a()'
The lexical_subs feature is experimental at -e line 1.
Assertion failed: (hek), function Perl_ck_subr, file op.c, line 10558.
Abort trap: 6
$ ./miniperl -Ilib -Mfeature=:all -e 'my sub a { is ref } a()'
The lexical_subs feature is experimental at -e line 1.
Assertion failed: (SvTYPE(_svmagic) >= SVt_PVMG), function S_mg_findext_flags, file mg.c, line 398.
Abort trap: 6
$

The prototype CV for a my sub is stored in magic attached to the pad
name.  The.  The code to fetch the prototype CV for a my sub calls
mg_find on the pad name.  If a syntax error occurs when the sub is be
ing compiled, the magic will never be attached, so the pad name (pad
names are currently SVs) will not have been upgraded to SVt_PVMG,
causing an assertion failure in mg_find, which only accepts SVs
thus upgraded.

When a pad entry is created, it is automatically filled with an empty
SV of the appropriate type.  For a subroutine, this is a nameless CV
stub.  CVs can be named in two ways, via GVs for package subs, or via
heks for lexical subs.  This stub has neither and is truly nameless.
Since a lexical sub is never installed if it contains a syntax error,
this stub is visible during subsequent compilation in the same scope.
ck_subr wasn’t prepared to handle a stub with absolutely no name
attached to it, since it is designed for handling sub calls where the
sub is known at compile time, so there must be a GV available to it,
unless the sub is lexical, and all lexical subs have heks.

This commit fixes the assumptions in both places.  Exactly what hap-
pens and what is returned is not so important, as this only hap-
pens after a syntax error, when the op tree is going to be thrown
away anyway.
make more use of regmatch_info struct.

regmatch_info is a small struct that is currently directly allocated as a
local var in Perl_regexec_flags(), and has a few fields that maintain part
of the state of the current pattern match. It is passed as an arg to
various functions that regexec_flags() calls, such as regtry().

In some ways its a rival to PL_reg_state, which also maintains state for
the current match, but which is a global variable (whose state needs
saving and restoring whenever the regex engine goes reentrant). It makes
more sense to store state in the regmatch_info struct, and as a first step
in moving more state to there, this commit makes more use of
regmatch_info.

In particular, it makes Perl_re_intuit_start() also allocate such a
struct, so that now *both* the main execution entry points to the regex
engine make use of it. It's also now passed as an arg to more of the static
functions that these two op-level ones call.

Two changes of special note. First, whether S_find_byclass() got called
with a null reginfo pointer of not indicated whether it had been called
from Perl_regexec_flags() (with a valid reginfo pointer), or from
Perl_re_intuit_start() (null pointer). Since they both pass non-null
reginfo pointers now, instead we add an extra field, reginfo->intuit that
indicates who's the top-level caller.

Secondly, to allow in future for various macros to uniformly refer to
values like reginfo->foo, where the structure is actually allocated as a
local var in Perl_regexec_flags(), we change the reginfo from being the
struct itself to being a pointer to the struct, (so Perl_regexec_flags
itself now uses reginfo->foo too rather than reginfo.foo).

In summary, all the above is essentially window dressing that makes no
functional changes to the code, but will facilitate future changes.
eliminiate PL_regeol

This is another global regex state variable (actually a field of
PL_reg_state). Eliminate it by moving it into the regmatch_info struct
instead, which is local to each match. Also, rename it to strend, which is
a less misleading description in these exciting days of multi-line matches.
find_byclass, regrepeat: remove is_utf8_pat arg

Remove the is_utf8_pat arg from these two static functions in regexec.c.
Since both these functions are now passed a valid reginfo pointer, this
info is already available as one of the fields in that struct.
add strbeg argument to Perl_re_intuit_start()

(note that this is a change both to the perl API and the regex engine
plugin API).

Currently, Perl_re_intuit_start() is passed an SV, plus pointers to:
where in the string to start matching (strpos); and to the end of the
string (strend).

Unlike Perl_regexec_flags(), it doesn't also have a strbeg arg.
Because of this this, it guesses strbeg: based on the passed SV (if its
svPOK()); or just set to strpos otherwise. This latter can happen if for
example the SV is overloaded. Note also that this latter guess is wrong,
and could in theory make /\b.../ fail.

But just to confuse matters, although Perl_re_intuit_start() itself uses
its guesstimate strbeg var, some of the functions it calls use the global
value of PL_bostr instead. To make this work, the *callers* of
Perl_re_intuit_start() currently set PL_bostr first. This is why \b
doesn't actually break.

The fix to this unholy mess is to simply add a strbeg arg to
Perl_re_intuit_start(). It's also the first step to eliminating PL_bostr
altogether.
eliminate PL_bostr

by moving it from the global PL_reg_state struct to the local reginfo
struct.
remove unused reginfo->bol field
S_regtry(): move eval setup code into separate fn

There's a block of code in S_regtry() that looks a bit like:

    if ((prog->extflags & RXf_EVAL_SEEN) && not_yet_done)
    {
	...
    }

Move this block of code out into a separate static function,
S_setup_eval_state(). No functional changes.

Also, rename the corresponding static cleanup/destructor function from
restore_pos() to S_restore_eval_state(), to better reflect what it does
these days (restoring pos() being only a small part of it).
S_setup_eval_state(): always set up destructor

This function only set up a destructor if reginfo->sv existed. If not,
some stuff which was set up wouldn't be restored if the code died within a
(?{}) block.

As it happens, the way the regex engine is called from perl core means we
always pass a valid SV; but in theory someone could call the engine from
XS while passing just a string and no SV.
add regmatch_eval_state struct

Replace several PL_reg* vars with a new struct. This is part of the
goal of removing all global regex state.

These particular vars are used in the case of a regex with (?{}) code
blocks. In this case, when the code in a block is called, various bits of
state (such as $1, pos($_)) are temporarily set up, even though the match
has not yet completed.

This involves updating the current PL_curpm to point to a fake PMOP which
points to the regex currently being executed. That regex has all its
current fields that are associated with captures (such as subbeg)
temporarily saved and overwritten with the current partial match results.
Similarly, $_ is temporarily aliased to the current match string, and any
old pos() position is saved. This saving was formerly done to the various
PL_reg* vars.

When the regex has finished executing (or if the code block croaks),
its fields are restored to the original values. Since this can happen in a
croak, it may be done using SAVEDESTRUCTOR_X() on the save stack. This
precludes just moving the PL_reg* vars into the regmatch_info struct,
since that is just allocated as a local var in regexec_flags(), and would
have already been abandoned and possibly overwritten after the croak and
longjmp, but before the SAVEDESTRUCTOR_X() action is taken.

So instead we put all the vars into new struct, and malloc that on entry to
the regex engine when we know we need to copy the various fields.
We save a pointer to that in the regmatch_info struct, as well as passing
it to SAVEDESTRUCTOR_X(). The destructor may get called up to twice in the
non-croak case: first it's called explicitly at the end of regexec_flags(),
which restores subbeg etc; then again from the savestack, which just
free()s the struct. In the croak case, it's called just once, and does
both the restoring and the freeing.

The vars / PL_reg_state fields this commit eliminates are:

    re_state_eval_setup_done
    PL_reg_oldsaved
    PL_reg_oldsavedlen
    PL_reg_oldsavedoffset
    PL_reg_oldsavedcoffset
    PL_reg_magic
    PL_reg_oldpos
    PL_nrs
    PL_reg_oldcurpm
stop callers of rex engine using RX_MATCH_UTF8_set

The way that the regex engine knows that the match string is utf8 is
currently a complete mess. It's partially signalled by the utf8 flag of
the passed SV, but also by the RXf_MATCH_UTF8 flag in the regex itself,
and the value of PL_reg_match_utf8.

Currently all the callers of the engine (such as pp_match, pp_split etc)
initially use RX_MATCH_UTF8_set() before calling the engine. This sets both
the RXf_MATCH_UTF8 flag on the regex, and PL_reg_match_utf8.

Then the two entry points to the engine (regexec_flags() and
re_intuit_start()) initially repeat the RX_MATCH_UTF8_set()
themselves.

Remove the usage of RX_MATCH_UTF8_set() by the callers of the engine,
and instead just rely on the engine to do it.

Also, remove the "secret" setting of PL_reg_match_utf8 by
RX_MATCH_UTF8_set(), and do it explicitly.

This is a prelude to eliminating PL_reg_match_utf8.
regex engine: simplify is_utf8_pat handling

Since this value is actually just always equal to cBOOL(RX_UTF8(rx)),
there's no need to save the old value of the local boolean
(as u.eval.saved_utf8_pat) when switching back and forwards between
regexes with (??{}); instead, just re-calculate it whenever we switch,
and update reginfo->is_utf8_pat and its cached value in the is_utf8_pat
local var accordingly.

Also, pass reginfo as an arg to S_setup_EXACTISH_ST_c1_c2() rather than
is_utf8_pat directly; this will allow us to eliminate PL_reg_match_utf8
shortly.

A new test is included that detects a mistake I made while working up
this change: I recalculated is_utf8_pat, but forgot to update
reginfo->is_utf8_pat too.
Eliminate PL_reg_match_utf8

Earlier commits made the use of this var just local to the current
match, so move it to the local regmatch_info struct instead.
S_setup_eval_state: save subcoffset, not suboffset

A typo in this function meant that the same value rex->suboffset was
saved to both eval_state->suboffset and eval_state->subcoffset.

However, I can't think of a test case where this will fail, since each
qr// gets its own set of suboffset vars these days. (and thus I wonder
whether there's even any need to save them when calling (?{}).