adamkennedy/PPI — pull request #72 by guillaumeaubert (2014-03-22 20:37:50+00:00)
Title: Fix parsing here-docs without a carriage return after the terminator

### Diagnostic

```PPI::Token::HereDoc``` notes that:

> If the last line matches the terminator but is missing the newline, we want to allow it anyway (like perl itself does).

In the ```__TOKENIZER__on_char()``` subroutine, the parsing of this use case is handled by this conditional block, which requires ```$line``` to be defined:

if ( defined $line and $line eq $token->{_terminator} ) {

However, because of the condition in the while loop that precedes that block, ```$line``` will _always_ be undefined after the while loop finishes:

while ( defined($line = $t->_get_line) ) {

The current code then considers the terminator without trailing newline to be part of the content of an unterminated here-doc, which is incorrect per the original comment in the code.

### Pull Request

This pull request provides the following:
* Use ```$token->{_heredoc}->[-1]``` instead of ```$line``` to detect terminators at the end of the file. ```$token->{_heredoc}->[-1]``` will always contain the last line of the file and this is what needs to be considered for parsing of the trailing terminator without carriage return afterwards.
* Scope ```$line``` to the while() loop only, since it is not used outside of it anymore.
* Remove the extra space added by ```PPI::Tokenizer``` at the end of the PPI document, when the here-doc block is at the end of the file. It was already correctly removed for unterminated here-docs, but it needs to be removed as well for here-docs with a proper terminator but without a carriage return at the end. This also allows the ```$token->{_heredoc}->[-1] eq $token->{_terminator}``` comparison to work properly.
* A complete suite of tests for the different cases (terminators with a trailing return, terminators without a trailing return, incorrectly terminated here-docs) across the various types of here-doc blocks (single quoted, double quoted, not quoted, commands, escaped).

Thank you!