snoyberg/xml — comment on commit f3d712cbfa76fe84567c6ff3a56eb994ef93c1fe
File:  line 0
By kawu:

I am not sure if this is the same problem, but when I used xml-conduit on UTF-8 encoded input I sometimes got the "Failed reading: takeWhile1" parse error. So I suppose it is somehow related. I don't know exactly where the problem is, but I've found some workaround solution which doesn't harm efficiency so much:

diff --git a/xml-conduit/Text/XML/Stream/Parse.hs b/xml-conduit/Text/XML/Stream/Parse.hs
index ffd291b..0f71102 100644
--- a/xml-conduit/Text/XML/Stream/Parse.hs
+++ b/xml-conduit/Text/XML/Stream/Parse.hs
@@ -185,8 +185,11 @@ detectUtf = C.conduitState
Left x -> return (Left x, C.Producing [])
Right (bss', decode) -> push (Right decode) bss'
push (Right decode) bss = do
- t <- C.conduitPush decode bss
+ t <- f <$> C.conduitPush decode bss
return (Right decode, t)
+ where
+ f (C.Producing x) = C.Producing $ filter (not . T.null) x
+ f x = x

close _ = return []

The idea is that C.conduitPush sometimes returns C.Producing value with empty chunk (is it not a bug ?). Filtering it out solves the problem (at least for me). It also looks like the problem occurs when multibyte characters are divided between different chunks.