Have CanonicalizePath handle \ on Windows

Have CanonicalizePath handle \ on Windows. It's slightly slower to do so, so compensate for that by improving ReadFile to not use text mode (which is a bit slower), and then fixing the lexer to accept \r (it was previously relying on reading files in text mode to remove '\r's).

Timings:

before patch:
qpc: 1229494us
qpc: 1232420us
qpc: 1232493us
qpc: 1235035us
qpc: 1239521us
qpc: 1247100us
avg = 1234867us

with canon of \, fixing parser to accept \r and speedier ReadFile
qpc: 1196304us
qpc: 1200643us
qpc: 1187478us
qpc: 1190062us
qpc: 1182991us
qpc: 1203037us
avg = 1190128us

on linux:

before:
real 0m0.675s
real 0m0.681s
real 0m0.698s
real 0m0.711s


after:
real 0m0.672s
real 0m0.680s
real 0m0.687s
real 0m0.706s

On Wed, Oct 29, 2014 at 9:12 PM, Nico Weber <notifications@github.com>
wrote:

> In src/lexer.in.cc:
>
> > @@ -132,8 +130,9 @@ Lexer::Token Lexer::ReadToken() {
> >      simple_varname = [a-zA-Z0-9_-]+;
> >      varname = [a-zA-Z0-9_.-]+;
> >
> > -    [ ]*"#"[^\000\r\n]*"\n" { continue; }
> > -    [ ]*[\n]   { token = NEWLINE;  break; }
> > +    [ ]*"#"[^\000\n]*"\n" { continue; }
> > +    [ ]*"\r\n" { token = NEWLINE;  break; }
>
> can you check that this doesn't change parsing speed on linux or mac?
> (checking one of those is enough). or i guess i can check myself.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/martine/ninja/pull/839/files#r19587174>.
>