racket/gui gtk: fix -singleInstance mode and relative paths
racket/gui gtk: fix -singleInstance for paths that don't work as strings

Just in case a relative path turned into an absolute path doesn't
work as a string, add support for a bytes encoding.