puppetlabs/puppet — pull request #2386 by joshcooper (2014-02-25 07:09:02+00:00)
Title: (PUP-1707) Ensure ERB generated help content is UTF-8 encoded

Previously, `env LANG=C bundle exec puppet man module` could not output
non US-ASCII characters correctly due to two issues.

First, puppet uses ERB generate the help output. Since the templates did
not include a magic encoding comment[1], ERB would set the string
encoding of the content it generated to `Encoding.default_external`, which
in the case of LANG=C and ruby 1.9 is US-ASCII.

When generating ERB help output for the `module` application, the resulting
byte stream would be correctly UTF-8 encoded, but `String#encoding` would
report US-ASCII. So later when we tried to perform a regex against it, we'd
get the `invalid byte sequence in US-ASCII` error.

The second issue is that the `man` application uses `IO.popen` to shell out
to the pager, but it did not specify the encoding for bytes it was writing
to the child process' stdin.

This commit sets the ERB magic encoding comment, and specifies an explicit
UTF-8 encoding when invoking the pager.

[1] https://github.com/ruby/ruby/blob/v1_9_3_484/lib/erb.rb#L63-L68