andris9/NodePie — Issue #9: decoding of invalid HTML entities replaces arbitrary chunks of content with a null character
Author: ryanmeador
Created: 2011-12-04 04:56:29+00:00
State: open

By default, nodepie decodes HTML entities in every(?) string that it returns through getContents(),getDescription() etc. I can't think of any reason why you would want to do that, because the XML parser will decode those fields when you parse the XML document. Nodepie then tries to decode them again, and if it encounters any ampersands, it assumes it is the beginning of an HTML entity. It will read until the next semicolon and then try to decode whatever it read. This is garbage, of course, so it doesn't decode properly, and is replaced with a null character instead. Then parsing resumes from where the semicolon was... so we have lost all the content between the ampersand and the semicolon, and we've replaced it with a null, which is not valid UTF8.

The workaround: turn on the option "keepHTMLEntities", which resolves the issue. Why isn't this on by default? Even if it was on by default, the behavior for decoding invalid HTML entities still needs to be fixed.