yihuang/tagstream-conduit — pull request #11 by chrisdone (2014-01-28 14:21:17+00:00)
Title: Entity decoding in Text tokens

Now the decode function for each will decode HTML entities:

λ> Text.HTML.TagStream.ByteString.decode "foo bar "mu< zot &hello;"
Right [Text "foo bar \"mu< zot "]
λ> Text.HTML.TagStream.Text.decode "foo bar "mu< zot &hello;"
Right [Text "foo bar \"mu< zot "]

The conduit (thanks to @snoyberg) also works:

λ> CL.sourceList (T.chunksOf 1 ">") C.$= T.tokenStream C.$$ CL.consume
[Text ">"]

In the spirit of tagstream-conduit's liberal parsing, invalid or unknown entities are skipped:

λ> Text.HTML.TagStream.Text.decode "&foo; &bar "mu< zot &hello;"
Right [Text " \"mu< zot "]

A test case has been added.