brianc/node-postgres — pull request #523 by mrkishi (2014-02-20 09:58:18+00:00)
Title: Add SQL_ASCII support and fix client_encoding inconsistency

Hello,

I'm working with a 3rd-party database that (unfortunately) uses the SQL_ASCII character set.
node-postgres presented problems decoding strings and after some research I think I came up with an acceptable solution. Note that I'm in no way a Pg expert, so please point out any problems!

SQL_ASCII is not supported because Connection (both native and javascript) always decodes strings as UTF-8. This isn't a problem with other encodings because PostgreSQL automatically converts strings in compatible charsets to the requested client_encoding, but it can result in invalid code points for SQL_ASCII because they are sent without conversion.

On top of that, the handling of client_encoding should be reviewed. The javascript version always use UTF-8 whereas the native bindings accept a configurable parameter. But without a proper decoding algorithm, there's no reason to accept different client_encodings (you just end up with an invalid UTF-8 string -- bug introduced in #370).

I see three possible solutions:

- Accept a configurable client_encoding and implement a decoder for [every Pg charset](http://www.postgresql.org/docs/9.3/static/multibyte.html).
- I can't see any upsides. Receive strings in a specific encoding just to decode them in javascript? Pg can convert them for us, probably faster.

- Accept a configurable client_encoding and leave strings as-is (ie. Buffer) if they are not UTF-8.
- Still easy to use in most cases (UTF-8).
- The user can request the original encoding if desired (to write to disk/network, decode manually for some reason, etc).

- Make client_connection fixed and let Pg convert strings for us (related issue: #498)
- Easiest of them all: just have to deal with the SQL_ASCII exception.
- The user can still encode them to different encodings if needed.

What do you think? I'm personally leaning torwards the second option. It'd work exactly the same as now, unless the user explicitly asks for the raw strings.

In this pull request I went with the third option, mainly because I'm not familiar with Node's C++ API and the native bindings seemed trickier to mess with.

For this, I centralized the client_encoding to solve the native/javascript disparity and added a check for the server_enconding runtime variable that's sent during the startup phase. If it is SQL_ASCII, I disable the UTF-8 decoding. Otherwise, leave it as before.

It still works the same, unless the user or server sets server_encoding to SQL_ASCII.

Sorry for the verbosity; being one of my first open-source contributions certainly didn't help that!