Repository: ninja-build/ninja (C++)

Issue:
windows device paths are handled incorrectly in depfiles
I'm using cmake with corrosion and cbindgen -- cbindgen creates gcc style dependency files, for which cmake sets "deps = gcc".  It does this since cmake version 3.29 (or at least 3.28 does not do this).

The generated depfiles use windows device paths, i.e. \\?\C:\path\to\file.

cmake's cmake_transform_depfile changes these to use forward slashes, i.e. //?/C:/path/to/file.

When these are added to .ninja_deps, the //? prefix is stripped and /C:/path/to/file can be seen in the .ninja_deps file.

Subsequent ninja runs fail with "ninja: error: FindFirstFileExA(/C:/path/to): ..."

The build.ninja file below is a standalone example showing the failure.  It needs a template.txt file in the same directory with some text.

The 1st COMMAND line shows the handling of a depfile matching what is produced by cmake's cmake_transform_depfile (i.e. containing paths with forward slashes).

The 2nd COMMAND line (commented out) shows the handling if the cmake step what skipped and the depfile contains paths with backslashes.  Note that ninja adds these paths as //?/C:/path/to/file, but doesn't strip a prefix.

\\?\C:\path\to\file is a valid pathname, and cmake arguably turns this into a less valid pathname, but since ninja ultimately stores the path with /, it would be good if it handled depfiles with slashes correctly.

```
rule CUSTOM_COMMAND
  command = $COMMAND

build output.txt: CUSTOM_COMMAND
  COMMAND = cmd.exe /c "copy template.txt output.txt& echo output.txt: //?/%CD:\=/%/template.txt >test.d"
  #COMMAND = cmd.exe /c "copy template.txt output.txt& echo output.txt: \\?\%CD%\template.txt >test.d"
  depfile = test.d
  deps = gcc
  restat = 1
```


Discussion: Thanks for reporting this, I can reproduce something like this on Linux with the following `build.ninja` file:

```ninja
rule output
  command = touch $out && echo "$out: //?/C:/path/to/input" > $out.d
  depfile = $out.d
  deps = gcc
  restat = 1

build output.txt: output

default output.txt
```

Using the following commands:

```bash
$ ninja
[1/1] touch output.txt && echo "output.txt: //?/C:/path/to/input" > output.txt.d

$ ninja -t deps
output.txt: #deps 2, deps mtime 1738673321279399520 (VALID)
    /
    /C:/path/to/input
```

The problem seems to be that the depfile parser (which has no Win32 specific code) treats the `?` as an input path separator, and thus splits it into `//` (which is later canonicalized to / but only on Linux) and `/C:/path/to/input`.

The culprit seems to be [this line](https://github.com/ninja-build/ninja/blob/7a988b5349357df78fe04c8e0dfa9c9d3d50979e/src/depfile_parser.in.cc#L137) which does not handle `?` in its regular expression.
Even when fixing this, another issue is that, since the path in the depfile uses forward slashes, the exact same path will be passed to `FindFirstFileA()` and other Win32 API functions.

I do not know if these accept forward slashes for root local device paths (i.e. those beginning with `\\?\` according to  https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html).
Thanks David -- I was just about to also point to the regex, and adding ? to the regex makes a correct entry in the deps log.

It's a bit of a weird situation with ? not a valid character in a Windows filename, except as part of `\\?\`.

For Linux, the regex seems a bit restrictive since ? (and quite a few other excluded characters) are valid, and gcc (`touch a\?fool.c && cc -c a\?fool.c -MMD -MF test.d`) emits dependency files like this:
```
$ cat test.d
a?fool.o: a?fool.c
```
which gets parsed incorrectly as "a" and "fool.c".

re FindFirstFileA() -- if the depfile contains a `\\?\` path with backslashes, and ninja puts it into its deps log with forward slashes, FindFirstFileA() doesn't fail like it does when the path is `/C:/path/to/file`.

FWIW, with the forward slash filename //?/C:/path/to/file in .ninja_deps, the dependency does work:
```
C:\tmp\ninja-test>ninja -t deps
output.txt: #deps 1, deps mtime 7603616931094012 (VALID)
    //?/C:/tmp/ninja-test/template.txt


C:\tmp\ninja-test>ninja
ninja: no work to do.

C:\tmp\ninja-test>echo check > template.txt

C:\tmp\ninja-test>ninja
[1/1] cmd.exe /c "copy template.txt output.txt& echo output.txt: \\?\%CD%\template.txt >test.d"
        1 file(s) copied.
```




Fix (patch):
diff --git a/src/depfile_parser.cc b/src/depfile_parser.cc
index 7ce7290614..9f7b393b4b 100644
--- a/src/depfile_parser.cc
+++ b/src/depfile_parser.cc
@@ -77,7 +77,7 @@ bool DepfileParser::Parse(string* content, string* err) {
           0, 128,   0,   0,   0, 128,   0,   0, 
         128, 128,   0, 128, 128, 128, 128, 128, 
         128, 128, 128, 128, 128, 128, 128, 128, 
-        128, 128, 128,   0,   0, 128,   0,   0, 
+        128, 128, 128,   0,   0, 128,   0, 128, 
         128, 128, 128, 128, 128, 128, 128, 128, 
         128, 128, 128, 128, 128, 128, 128, 128, 
         128, 128, 128, 128, 128, 128, 128, 128, 
@@ -105,55 +105,55 @@ bool DepfileParser::Parse(string* content, string* err) {
       };
       yych = *in;
       if (yybm[0+yych] & 128) {
-        goto yy9;
+        goto yy5;
       }
       if (yych <= '\r') {
         if (yych <= '\t') {
-          if (yych >= 0x01) goto yy4;
+          if (yych >= 0x01) goto yy1;
         } else {
-          if (yych <= '\n') goto yy6;
-          if (yych <= '\f') goto yy4;
-          goto yy8;
+          if (yych <= '\n') goto yy3;
+          if (yych <= '\f') goto yy1;
+          goto yy4;
         }
       } else {
         if (yych <= '$') {
-          if (yych <= '#') goto yy4;
-          goto yy12;
+          if (yych <= '#') goto yy1;
+          goto yy7;
         } else {
-          if (yych <= '?') goto yy4;
-          if (yych <= '\\') goto yy13;
-          goto yy4;
+          if (yych <= '>') goto yy1;
+          if (yych <= '\\') goto yy8;
+          goto yy1;
         }
       }
       ++in;
       {
         break;
       }
-yy4:
+yy1:
       ++in;
-yy5:
+yy2:
       {
         // For any other character (e.g. whitespace), swallow it here,
         // allowing the outer logic to loop around again.
         break;
       }
-yy6:
+yy3:
       ++in;
       {
         // A newline ends the current file name and the current rule.
         have_newline = true;
         break;
       }
-yy8:
+yy4:
       yych = *++in;
-      if (yych == '\n') goto yy6;
-      goto yy5;
-yy9:
+      if (yych == '\n') goto yy3;
+      goto yy2;
+yy5:
       yych = *++in;
       if (yybm[0+yych] & 128) {
-        goto yy9;
+        goto yy5;
       }
-yy11:
+yy6:
       {
         // Got a span of plain text.
         int len = (int)(in - start);
@@ -163,54 +163,54 @@ bool DepfileParser::Parse(string* content, string* err) {
         out += len;
         continue;
       }
-yy12:
+yy7:
       yych = *++in;
-      if (yych == '$') goto yy14;
-      goto yy5;
-yy13:
+      if (yych == '$') goto yy9;
+      goto yy2;
+yy8:
       yych = *(yymarker = ++in);
       if (yych <= ' ') {
         if (yych <= '\n') {
-          if (yych <= 0x00) goto yy5;
-          if (yych <= '\t') goto yy16;
-          goto yy17;
+          if (yych <= 0x00) goto yy2;
+          if (yych <= '\t') goto yy10;
+          goto yy11;
         } else {
-          if (yych == '\r') goto yy19;
-          if (yych <= 0x1F) goto yy16;
-          goto yy21;
+          if (yych == '\r') goto yy12;
+          if (yych <= 0x1F) goto yy10;
+          goto yy13;
         }
       } else {
         if (yych <= '9') {
-          if (yych == '#') goto yy23;
-          goto yy16;
+          if (yych == '#') goto yy14;
+          goto yy10;
         } else {
-          if (yych <= ':') goto yy25;
-          if (yych == '\\') goto yy27;
-          goto yy16;
+          if (yych <= ':') goto yy15;
+          if (yych == '\\') goto yy17;
+          goto yy10;
         }
       }
-yy14:
+yy9:
       ++in;
       {
         // De-escape dollar character.
         *out++ = '$';
         continue;
       }
-yy16:
+yy10:
       ++in;
-      goto yy11;
-yy17:
+      goto yy6;
+yy11:
       ++in;
       {
         // A line continuation ends the current file name.
         break;
       }
-yy19:
+yy12:
       yych = *++in;
-      if (yych == '\n') goto yy17;
+      if (yych == '\n') goto yy11;
       in = yymarker;
-      goto yy5;
-yy21:
+      goto yy2;
+yy13:
       ++in;
       {
         // 2N+1 backslashes plus space -> N backslashes plus space.
@@ -222,7 +222,7 @@ bool DepfileParser::Parse(string* content, string* err) {
         *out++ = ' ';
         continue;
       }
-yy23:
+yy14:
       ++in;
       {
         // De-escape hash sign, but preserve other leading backslashes.
@@ -233,17 +233,17 @@ bool DepfileParser::Parse(string* content, string* err) {
         *out++ = '#';
         continue;
       }
-yy25:
+yy15:
       yych = *++in;
       if (yych <= '\f') {
-        if (yych <= 0x00) goto yy28;
-        if (yych <= 0x08) goto yy26;
-        if (yych <= '\n') goto yy28;
+        if (yych <= 0x00) goto yy18;
+        if (yych <= 0x08) goto yy16;
+        if (yych <= '\n') goto yy18;
       } else {
-        if (yych <= '\r') goto yy28;
-        if (yych == ' ') goto yy28;
+        if (yych <= '\r') goto yy18;
+        if (yych == ' ') goto yy18;
       }
-yy26:
+yy16:
       {
         // De-escape colon sign, but preserve other leading backslashes.
         // Regular expression uses lookahead to make sure that no whitespace
@@ -255,29 +255,29 @@ bool DepfileParser::Parse(string* content, string* err) {
         *out++ = ':';
         continue;
       }
-yy27:
+yy17:
       yych = *++in;
       if (yych <= ' ') {
         if (yych <= '\n') {
-          if (yych <= 0x00) goto yy11;
-          if (yych <= '\t') goto yy16;
-          goto yy11;
+          if (yych <= 0x00) goto yy6;
+          if (yych <= '\t') goto yy10;
+          goto yy6;
         } else {
-          if (yych == '\r') goto yy11;
-          if (yych <= 0x1F) goto yy16;
-          goto yy30;
+          if (yych == '\r') goto yy6;
+          if (yych <= 0x1F) goto yy10;
+          goto yy19;
         }
       } else {
         if (yych <= '9') {
-          if (yych == '#') goto yy23;
-          goto yy16;
+          if (yych == '#') goto yy14;
+          goto yy10;
         } else {
-          if (yych <= ':') goto yy25;
-          if (yych == '\\') goto yy32;
-          goto yy16;
+          if (yych <= ':') goto yy15;
+          if (yych == '\\') goto yy20;
+          goto yy10;
         }
       }
-yy28:
+yy18:
       ++in;
       {
         // Backslash followed by : and whitespace.
@@ -291,7 +291,7 @@ bool DepfileParser::Parse(string* content, string* err) {
           have_newline = true;
         break;
       }
-yy30:
+yy19:
       ++in;
       {
         // 2N backslashes plus space -> 2N backslashes, end of filename.
@@ -301,26 +301,26 @@ bool DepfileParser::Parse(string* content, string* err) {
         out += len - 1;
         break;
       }
-yy32:
+yy20:
       yych = *++in;
       if (yych <= ' ') {
         if (yych <= '\n') {
-          if (yych <= 0x00) goto yy11;
-          if (yych <= '\t') goto yy16;
-          goto yy11;
+          if (yych <= 0x00) goto yy6;
+          if (yych <= '\t') goto yy10;
+          goto yy6;
         } else {
-          if (yych == '\r') goto yy11;
-          if (yych <= 0x1F) goto yy16;
-          goto yy21;
+          if (yych == '\r') goto yy6;
+          if (yych <= 0x1F) goto yy10;
+          goto yy13;
         }
       } else {
         if (yych <= '9') {
-          if (yych == '#') goto yy23;
-          goto yy16;
+          if (yych == '#') goto yy14;
+          goto yy10;
         } else {
-          if (yych <= ':') goto yy25;
-          if (yych == '\\') goto yy27;
-          goto yy16;
+          if (yych <= ':') goto yy15;
+          if (yych == '\\') goto yy17;
+          goto yy10;
         }
       }
     }
diff --git a/src/depfile_parser.in.cc b/src/depfile_parser.in.cc
index 4b5f5fe4c1..5bef2585bf 100644
--- a/src/depfile_parser.in.cc
+++ b/src/depfile_parser.in.cc
@@ -134,7 +134,7 @@ bool DepfileParser::Parse(string* content, string* err) {
         *out++ = '$';
         continue;
       }
-      '\\'+ [^\000\r\n] | [a-zA-Z0-9+,/_:.~()}{%=@\x5B\x5D!\x80-\xFF-]+ {
+      '\\'+ [^\000\r\n] | [a-zA-Z0-9+?,/_:.~()}{%=@\x5B\x5D!\x80-\xFF-]+ {
         // Got a span of plain text.
         int len = (int)(in - start);
         // Need to shift it over if we're overwriting backslashes.