aliviate some of the nastiness of the UTF-8 support patch.

reduce repeated code and encapsulate UTF8 specific behavior to where it is used.