> Would this introduce a performance hit? Now you have to decode the entire string.

Yes, it would introduce a performance hit, but there's no way around it: We're either unicode correct, or we aren't.

> I wonder if it's possible to restructure the code to have separate loops

I tried it. *VERY* aggressively too, but the "simple" algorithm is actually really fast in its simplicity.

> and skipping runs of whitespace. You might be able to get rid of the need to decode that way, perhaps?

There are unicode whitespaces too, so there's no way around the decode, either for whites or non-whites.