move unicode detection to a separate function

this will ease transition to a commandline option override