[perl #119505] Segfault from bad backreference

The code that parses regex backrefs (or ambiguous backref/octal) such as
\123, did a simple atoi(), which could wrap round to negative values on
long digit strings and cause seg faults.

Include a check on the length of the digit string, and if greater than 9
digits, assume it can never be a valid backref (obviating the need for the
atoi() call).

I've also simplified the code a bit, putting most of the \g handling code
into a single block, rather than doing multiple "if (isg) {...}".