Fix #3961 : use char range methods instead of byte offsets to detect whitespace.
checkpoint block-comment-wchar state.
make pp file conform to actual output, noting some oddities along the way.
fixed code to placate new restrictions on form of function/method invocations.
auto merge of #7114 : pnkfelix/rust/issue3961-fix-whitespace-detection, r=brson

r?  (yes, the review request is back, now that I got it building against incom... I mean master!)

(Attempting to port from orphaned pull-request #6764 )

Fix for #3961. Also includes a test case to illustrate the issues. (All of the entries that say "should align" should align with each other, and the four lines near the end that say "compare _" for _ in {A,B,C,D} should line up with each other.)

Before applying this change set:
-- the "(should align)"'s are all over the place, and the form/line feeding spaces are not cut out as one might or might not expect.
-- compare B and D do not match A and C.

(To be honest, its hard to really say what the right behavior is here, and people who are expecting a particular behavior out of a pretty printer in these cases may well get burned.)