rwstauner/Text-VimColor — Issue #1: Make it easier to work with encoded (non-ascii) strings
Author: rwstauner
Created: 2011-11-04 15:25:27+00:00
State: open

RJBS recently made this change:
https://github.com/rjbs/pod-elemental-transformer-vimhtml/commit/d465c21d5403cfd593129c34a66c5c6f7ac3f11a

Here is his report, excerpted from IRC:

https://skitch.com/rjbs/gfti4/advcal-unicode
should look like
https://skitch.com/rjbs/gfthb/advcal-unicode

Things to report:

1. T::VC expects "string" to be a byte string, not text. I had to encode it first.
2. T::VC returns a byte string, not a text string. I had to decode it afterward.
3. I had to communicate the character encoding of the bytestream to Vim, which meant passing '+set fenc=utf-8', but I could not do this without copying and pasting the default options as well.

Thoughts on addressing this:

1. Either document that it expects a byte string or fix it to expect unicode, which it will then encode, and document that it expects text, not bytes.
I suggest the latter. People passing a string should be passing unencoded text. It also eliminates the question of passing the encoding, because if you pass a byte string, you will also need to allow for the encoding to be passed.
2. I strongly suggest returning a character string, but otherwise document that an encoded byte string is returned. In either case, you probably need to pass an argument to ensure that it is always returned in one encoding, so it can be reliably decoded.
3. provide access to @VIM_OPTIONS (or whatever it was called) as ->default_vim_options so it can be included in "those options plus more" as (options => [ T::VC->default_vim_options, ... ])