Allow test_bootstrap.t to run from the top level
Use test.pl in lexsub.t

I thought cmd/ couldn’t use test.pl, but was mistaken.
lexsub.t: Fix a test

This is not testing what I meant it to test: that ‘sub d’ will respect
a preceding ‘our sub d;’.  If ‘sub d’ is in the same package, it makes
no difference, so the test tests nothing.

It turns out this does not work yet.
do.t: Load test.pl at BEGIN time

so that parentheses can be omitted.
Let do.t run from the top level
do-file should not force a bareword

A word following do is forced to be a bareword for do-sub’s sake.  But
if it is going to be interpreted as do-file after all, that does not
make sense.  ‘do subname;’ should call the sub and run the file whose
name it returns, instead of running the file named ‘subname’.
Make do sub() respect our declarations
Fix our sub with proto

yylex must emit exactly one token each time it is called.  Some-
times yylex needs to parse several tokens at once.  That’s what
the various force functions are for.  But that is also what
PL_pending_ident is for.

The various force_next, force_word, force_ident, etc., functions keep
a stack of tokens (PL_nextval/PL_nexttype) that yylex will check imme-
diately when called.

PL_pending_ident is used to track a single identifier that yylex will
hand off to S_pending_ident to handle.

S_pending_ident is the only piece of code for resolving an identi-
fier that could be lexical but could also be a package variable.
force_ident assumes it is looking for a package variable.

force_* takes precedence over PL_pending_ident.

All this means that, if an identifier needs to be looked up in the pad
on the next yylex invocation, it has to use PL_pending_ident, and the
force_* functions cannot be used at the same time.

Not realising that, when I made ‘our sub foo’ store the sub in the
pad I also made ‘our sub foo ($)’ into a syntax error, because it
was being parsed as ‘our sub ($) foo’ (the prototype being ‘forced’);
i.e., the pending tokens were being pulled out of the ‘queue’ in the
wrong order.  (I put queue in quotes, because one queue and one unre-
lated buffer together don’t exactly count as ‘a queue’.)

Changing PL_pending_ident to have precedence over the force stack
breaks ext/XS-APItest/t/swaptwostmts.t, because the statement-parsing
interface does not localise PL_pending_ident.  It could be changed to
do that, but I don’t think it is the right solution.

Having two separate pending token mechanisms makes things need-
lessly fragile.

This commit eliminates the PL_pending_ident mechanism and
modifies S_pending_ident (renaming it in the process to
S_force_ident_maybe_lex) to work with the force mechanism.  I was
going to merge it with force_ident, but the two make incompatible
assumptions that just complicate the code if merged.  S_pending_ident
needs the sigil in the same string buffer, to pass to the pad inter-
face.  force_ident needs to be able to work without a sigil present.

So now we only have one queue for pending tokens and the order is more
predictable.
Test initial tick in sub declaration
Make ‘sub foo{}’ respect ‘our foo’

This commit switches all sub definitions, whether with ‘our’ or not,
to using S_force_ident_maybe_lex (formerly known as S_pending_ident).

This means that an unqualified (no our/my/state or package prefix)
‘sub foo’ declaration does a pad lookup, just like $foo.

It turns out that the vivification that I added to the then
S_pending_ident for CVs was unnecessary and actually buggy.  We
*don’t* want to autovivify GVs for CVs, because they might be con-
stants or forward declarations, which are stored in a simpler form.

I also had to change the subname rule used by MYSUB in perly.y, since
it can now be fed a PRIVATEREF, which it does not expect.  This may
prove to be temporary, but it keeps current tests passing.
toke.c:yylex:KEY_sub can use PL_tokenbuf to begin with

There is no need to allocate a separate ‘tmpbuf’ and then copy it into
PL_tokenbuf afterwards.
Let barewords look up our subs

These take precedence over built-in keywords (just as my $AUTOLOAD
shadows the package var), but not the keyword plugin, as the latter
takes precedence over labels, and these don’t.
lexsub.t: Add test name, test override from another pkg

The bareword logic in toke.c looks up GVs in various places.  This
tests that we are bypassing those correctly.
Store state subs in the pad

In making ‘sub foo’ respect previous ‘our sub’ declarations in a
recent commit, I actually made ‘state sub foo’ into a syntax error.
(At the time, I patched up MYSUB in perly.y to keep the tests for ‘"my
sub" not yet implemented’ still working.)  Basically, it was creat-
ing an empty pad entry, but returning something that perly.y was not
expecting.

This commit adjusts the grammar to allow the SUB branch of barestmt to
accept a PRIVATEREF for its subname, in addition to a WORD.  It reuses
the subname rule that SUB used to use (before our subs were added),
gutting it to remove the special block handling, which SUB now tokes
care of.  That means the MYSUB rule will no longer turn on CvSPECIAL
on the PL_compcv that is going to be thrown away anyway.

The code for special blocks (BEGIN, END, etc.) that turns on CvSPECIAL
now checks for state subs and skips those.  It only applies to our
subs and package subs.

newMYSUB has now actually been written.  It basically duplicates
newATTRSUB, except for GV-specific things.  It does currently vivify a
GV and set CvGV, but I am hoping to change that later.  I also hope to
merge some of the code later, too.

I changed the prototype of newMYSUB to make it easier to use.  It is
not used anywhere on CPAN and has always simply died, so that should
be all right.