theory/pod-simple — pull request #40 by grantm (2012-08-27 01:44:07+00:00)
Title: encoding heuristic should be applied to first non-ASCII byte sequence

I've just realised that the heuristic in my patch to recognise a UTF8 byte sequence is not being applied in strict accordance with the documentation.

The intention was to analyse the first sequence of non-ASCII bytes however because I failed to anchor the regex, it will potentially look past the first non-ASCII byte and look for possible UTF8 sequences later in the line.

The failure mode would be incorrectly guessing UTF8 when the source was actually Latin-1. The chances of this happening in actual Latin-1 are still close to zero but if the source is CP1252 ("Win-Latin-1") and contains "smart quote" characters then it's a little more likely. Of course if we guess Latin-1 those smart quotes aren't going to be rendered correctly anyway.

Also the potential for error is limited to files which omit an =encoding declaration - since we don't try guessing at all if the encoding is explicitly declared.

All of which is to say although it's worth changing this so the implementation matches the documentation it's not an urgent patch.