html5lib/html5lib-python — pull request #162 by lastorset (2014-06-06 13:41:49+00:00)
Title: Replace invalid characters with U+FFFD (fixes #96)

This fix simply repeats the encoding-specific replacement with a general one using invalid_unicode_re. It corresponds to section 12.2.2.5. I can't quite tell what the spec says to do if one of these characters is encountered, but the rest of the spec replaces other characters with U+FFFD, so I did that (despite Simon's preference of the empty string).

I can submit a test for this (AFAICT I have to do that separately).