sub(){__SUB__} under -d, sub{eval"";__SUB__}

The peephole optimiser changes __SUB__ to a constant if the sub is not
clonable.  All anonymous subs are clonable under the debugger, but
they are not marked as such until after the peephole optimiser runs.

If a sub is clonable, then it undergoes a second search through the
op tree (op_const_sv) to see if it is inlinable.  Trying to make
sub(){__SUB__} inlinable results in some sort of memory corruption,
in which it presumably points to a freed SV that get reused for some-
thing else:

$ PERL5DB='sub DB::DB{}' ./perl -dIlib  -lE 'BEGIN{*f = sub(){__SUB__}}; print f; print \&f'
*IO::File::AUTOLOAD
CODE(0x7fef62035760)

(Both lines should show the same thing.)

The same clonability applies to subs containing a string eval.  The
sub is not marked clonable yet when __SUB__ becomes a constant, so it
ends up referring to the closure prototype:

$ ./perl -Ilib  -lE 'BEGIN{*f = sub(){eval""; __SUB__}}; print f; print \&f; f->()'
CODE(0x7fe8c2036fe8)
CODE(0x7fe8c2031730)
Closure prototype called at -e line 1.

When turning __SUB__ into a constant, we need to check the same con-
dition (PL_cv_has_eval || PL_perldb; see pad_tidy) that would make a
sub clonable.
Make sub(){$outer_lexical} return a copy each time

We don’t actually stop in from being inlined, but simply mark the
inlined value as PADTMP, so it gets copied in lvalue context.
Preserve :method in sub:method(){$outer_lex}

When we turn such a sub into a constant, we need to preserve the
attribute, since it changes behaviour.
Don’t turn sub:CustomAttr(){$outer_lex} into a const

We can’t know what the attributes are for, and turning it into a con-
stant discards the attributes.
Update comments about sub(){$x} consting

It no longer has anything to do with constant.pm.
Add new LVALUE flag for pad names

This will be used to record whether a pad entry is used as an lvalue
multiple times.  If so, it cannot be used as a constant.
op.c: Record lvalue use of lexicals

other than where the variable is declared.

This will be used to determine whether my $x; sub(){$x} can make a
constant.  Currently, this becomes a constant even if $x is subse-
quently modified, breaking the closure behaviour (bug #79908).
Don’t attempt to inline my sub (){$outer_var}

$ perl5.18.2 -Ilib  -Mfeature=lexical_subs -e ' my $x; my sub a(){$x}; print a'
The lexical_subs feature is experimental at -e line 1.
Segmentation fault: 11

Same in blead.

When calls to the sub are compiled (the ‘a’ in ‘print a’) the value
of the lexical variable cannot possibly known, because the sub hasn’t
been cloned yet, and all we have is the closure prototype.

A potentially constant closure prototype is marked CvCONST and
cv_const_sv_or_av (called by the code in toke.c that handles bare-
words) thinks that CvCONST means we have a constant XSUB.  Only lexi-
cal subs allow a closure prototype to reach that function.

We shouldn’t mark the closure prototype as CvCONST to begin with.
Because when we do, the ‘constant’ is retrieved from CvXUBANY, which
is a union shared by CvSTART.  Then toke.c does SvREFCNT_inc on
CvSTART, and screws up the op tree.
pad.c:S_cv_clone: Add assertion

This code cannot handle the case where cloning uses an existing
stub, because it creates a new CV via newCONSTSUB.  Cloning into
an existing stub only happens for lexical subs, and the previous
commit prevented them from reaching this code.  Before that we
would have had assertion failures in pp_padcv.
pad.c: Have S_cv_clone_pad return the CV

Currently it is a void function, because it modifies the CV in place.
Shortly it will sometimes return a different CV from the one it was
passed.
pad.c: Move constant closure code

S_cv_clone normally calls S_cv_clone_pad, which takes care of cloning
the pad, surprisingly enough.  Then S_cv_clone checks whether it can
turn a closure into a constant.  That code needs to be moved into
S_cv_clone_pad, because, not only is it pad-related, but to work cor-
rectly it needs to access the outer sub (which S_cv_clone_pad already
fetches for its own use), which future commits will make it do.
Put sub(){ ... } constant tests in a new file
Deprecate inlining sub(){$x} if $x is changed elsewhere

With the new PadnameLVALUE flag, we can detect cases where an outer
lexical is used multiple times in lvalue context.  If the subroutine
contains just a lexical variable and nothing else, deprecate this
behaviour, so we can change it not to break the closure pattern at
some future date.

Future commits will fix those cases where the subroutine contains more
than just a lexical variable, without a deprecation cycle.

Adding this code to op_const_sv doesn’t really make sense.  More to
the point, having S_cv_clone_pad call op_const_sv doesn’t make sense,
but changing that (moving this code directly to S_cv_clone_pad) will
require other refactorings to avoid breaking some cases of constant
(real constant)  inlining, such as sub(){$x++ if 0; 3}, which cur-
rently gets inlined.
Use ‘behavior’ consistently in perldiag

‘Behavior’ is used many times; ‘behaviour’ only three.
Restructure const-optree.t

This new layout (it is completely rewritten) allows endless variations
to be tested more easily.
Don’t inline sub(){0; $x} if $x changes elsewhere

Some op trees will turn a sub into a constant even if they
are more than just a simple constant or lexical variable.  In
particular, a statement optimised away followed by a lexi-
cal variable is eligible for inlining.  As discussed in
<20141101170924.17763.qmail@lists-nntp.develooper.com>, make these
more complex op trees follow closure rules properly.  If the outer
lexical is used in lvalue context in places other than its declara-
tion, then we should forego inlining.
Don’t inline sub(){my $x; state sub z {$x} $outer}

The code that determines whether a closure prototype is potentially eligi-
ble for inlining when cloned iterates through the op chain in the order it
is executed.  It looks for padsv (or const) followed by sub exit.  Cer-
tain ops with no side effects, like nextstate and null, are skipped over.
There is a flaw in the logic handling padsv ops.  If the padsv op closes
over a variable from outside, we record that we have seen an SV, so
another padsv or const prevents inlining.  If the first padsv op does
not close over a variable from outside, but belongs to this sub itself,
then we just ignore it and skip over it as we would a null op.  So that
means sub () { my $x; $outer_var } is marked as potentially eligible
for inlining.

Now, when cloning happens (i.e., when the ‘sub’ expression is evaluated),
we search through the op chain (once more) of subs marked as potentially
inlinable and find the first padsv op, which we assume is the one that
closes over the outer sub.  So we get the value of the wrong variable.

Now, there is a reference count check that usually saves the day.  The
reference count must be exactly 2, one reference held by the newly-cloned
closure and the other by the outer sub.  Usually ‘my $x’ will have a ref-
erence count of 1 when the sub is cloned, so it does not become inlina-
ble, but behaves as expected.

With state subs, however, which are cloned when the enclosing sub is
cloned, we can make that inner lexical have a reference count of 2.
So the sub becomes inlinable, using the undefined value taken from the
wrong variable:

$ ./perl -Ilib -MO=Concise -Mfeature=:all -e 'BEGIN { my $x = 43; *foo = sub :prototype(){my $y; state sub z { $y } $x}} print foo()'
The lexical_subs feature is experimental at -e line 1.
6  <@> leave[1 ref] vKP/REFC ->(end)
1     <0> enter ->2
2     <;> nextstate(main 53 -e:1) v:%,{,469764096 ->3
5     <@> print vK ->6
3        <0> pushmark s ->4
4        <$> const[NULL ] s*/FOLD ->5
-e syntax OK

Notice the const[NULL ], which indicates undef.

At least that’s what we get if we are lucky enough not to crash in the
padname-handling code added recently to op.c:op_const_sv.  Sometimes
this results:

$ ./perl -Ilib -MO=Concise -Mfeature=:all -e 'BEGIN { my $x = 43; *foo = sub :prototype(){my $y; state sub z { $y } $x}} print foo()'
The lexical_subs feature is experimental at -e line 1.
Segmentation fault: 11
const-optree.t: More tests

Test explicit return with variable (single statement) and simple scalar
not modified elsewhere.
Don’t inline sub(){ 0; return $x; ... }

We document that explicit return prevents subs from being inlined.
But if that explicit return comes after a statement optimised away,
then it does not prevent inlining.

Originally, explicit return did not prevent inlining, and the way
constant subs were detected was to search through the op chain in
execution order, looking for padsv followed by return or leavesub.
Later, things were accidentally changed, such that the search began,
not at the beginning of the execution chain, but at the outer-
most op of the first statement’s contents.  That means that, with
sub () { $a + $b }, we begin the search at the + (add), even though
the execution order is nextstate, gvsv, gvsv, add, leavesub.  So for
sub () { return $x } we begin the search at return, and miss the padsv
that precedes it.

Even though it was accidental that ‘return’ inlining was broken, it
ended up becoming a documented feature, and has been so for over a
decade.  So I am just making things consistent with that.

This commit only affects those cases where the return is not at the
end of the sub.  At the end of the sub, the return is optimised out of
the execution chain, so it requires a little more work, which the next
commit will do.
Don’t inline sub(){ 0; return $x }

If return occurs at the end of a sub, it is optimised out of the
execution, chain, so we have to look at the op tree structure to
detect it.