puppetlabs/puppetdb — pull request #261 by cprice-puppet (2012-08-22 01:45:04+00:00)
Title: Bug/master/15903 more fun with utf8

commit b0167a4b54d42db62407bcd5c9ce1efc4d6ff9d2
Author: Chris Price
Date: Tue Aug 21 18:43:58 2012 -0700

(#15903) Detect invalid multi-byte sequences starting with 0xF4

The UTF-8 spec says that codepoints greater than 0x10FFFF are
illegal. The first character that is over that limit is
0xF490bfbf, so if the first byte of a multibyte character is F4,
then we have to check for that condition. This commit adds that
check, along with corresponding tests.

commit 0e23b96b6de5cd38b0e2e2938dcade092eafacd5
Author: Chris Price
Date: Tue Aug 21 16:00:25 2012 -0700

(#15903) Detect invalid multi-byte sequences from 0xC0, 0xC1, 0xF5-0xFF

Prior to this commit, our UTF-8 sanitizer was not recognizing
multi-byte sequences starting with 0xC0, 0xC1, or anything in
the range of 0xF5-0xFF invalid. Java/Clojure *do* consider
these invalid (which matches up with the explanation on the
UTF-8 page on wikipedia). This commit adds detection for
those byte sequences.