joshfire/node-crawler — pull request #29 by headconnect (2012-10-30 19:12:00+00:00)
Title: Change from iconv to iconv-lite

It seems that I'm not the only one having difficulties dealing with iconv.

Proposing change from iconv to iconv-lite (https://github.com/ashtuchkin/iconv-lite)

Benefits:
- Cleaner code (one liner)
- Quicker conversion (though shouldn't be an issue anyway I'd think)

Drawbacks:
- Doesn't support 'Big5' charset (chinese)
- Also doesn't support: "EUC family, Shift_JIS." (as quoted from readme.md).

I don't believe the EUC family nor Shift_JIS will be likely to be encountered on the net, though Big5 /might/ be somewhat annoying for those crawling the east (I really don't know how widespread this is anyway). In any event, this will make deployments that much easier as it doesn't have any deeper platform requirements.

* Note: I've been trying to test it fully - got everything except 'errors.js' passing on cloud9 and centos 6 with nodejs 6.x and 8.x respectively, though got an error in 'forceutf8.js' when testing on windows. I'm thinking there's something related to catching invalid responses (each time it's 'cannot read property 'body' of undefined' errors). Worth checking out thouth I'd say.