glguy/utf8-string — Issue #3: isUTF8Encoded bug
Author: sergv
Created: 2011-10-24 22:04:36+00:00
State: open

Codec.Binary.UTF8.String.isUTF8Encoded reports false negative result for the test file with these 5 characters:

ØŠěáě

My test program:
'''haskell

module IsUTF8 where

import System.IO
import Codec.Binary.UTF8.String

main = getContents >>= putStrLn . show . isUTF8Encoded
'''

Tested like this:
sergey@debian:~/emacs/misc/obsolete/deprecated$ cat file
ØŠěáě
sergey@debian:~/emacs/misc/obsolete/deprecated$ ghc -o IsUTF8 -main-is IsUTF8 IsUTF8.hs
sergey@debian:~/emacs/misc/obsolete/deprecated$ cat file | ./IsUTF8
False
sergey@debian:~/emacs/misc/obsolete/deprecated$ cat file | isutf8

isutf8 (http://linux.die.net/man/1/isutf8) reports nothing, meaning file is valid. Editors are pretty convinced in this conclusion too.