Repository: bazelbuild/bazel
Issue: Use symlinked runfiles on Windows when supported
Labels: type: feature request
P2
platform: windows
area-Windows
team-OSS

### Description of the problem / feature request:

On Windows Bazel sets `RUNFILES_MANIFEST_ONLY` to `1` and does not generate symlinks to runfiles. This does not change if the current user can create symlinks.

On recent Windows 10 versions it is possible to enable symlinks for all users:

> Now in Windows 10 Creators Update, a user (with admin rights) can first enable Developer Mode, and then any user on the machine can run the mklink command without elevating a command-line console.

More details can be found in https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

The feature I am requesting is to generate symlink to runfiles on Windows when supported by the current user.

### Feature requests: what underlying problem are you trying to solve with this feature?

Creating rules with windows support is not trivial (https://github.com/bazelbuild/bazel/issues/3726#issuecomment-329504506). 

Using symlinks on windows when available should make many existing rules work on Windows without having to resolve runfiles via the manifest, given the user has symlink support. 

On systems Windows without symlink support enabled, looking up the runfiles would still be necessary.

Related to https://github.com/bazelbuild/bazel/issues/3726, https://github.com/bazelbuild/bazel/issues/3889, https://github.com/bazelbuild/bazel/issues/1212

/cc @alexeagle

Issue comments:
@filipesilva I have made some progress on this recently.
Now with Bazel built from HEAD, symlink tree will be available on Windows when you pass `--experimental_enable_runfiles`. 
Can you try if it helps with https://github.com/bazelbuild/bazel/issues/5926?
@filipesilva Just found a bug when dealing with long path, sending a fix now.

Pull request review comments:
`FILE_FLAG_BACKUP_SEMANTICS` is required when and only when opening a directory or junction. When opening a file symlink, you must not use `FILE_FLAG_BACKUP_SEMANTICS`. I suggest trying to open it with `FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OPEN_REPARSE_POINT` in case this `CreateFileW` call fails.

Furthermore, in order to minimize risk of filesystem races I suggest attempting to open the file without trying to call `GetFileAttributesW`. The benefit: if the path does not exist, `CreateFileW` fails and `GetLastError()` will return a corresponding error code, but if the path does exist, we have an open handle to it and no other process can delete or modify the file until we release this handle. (The OS will delay actual deletion until all handles are closed.)

And in case you do need the attributes after opening the file, you can use `GetFileInformationByHandle`.
Good suggestions! Thanks!
In my experience, CreateFileW(..., FILE_ATTRIBUTE_NORMAL | ...) fails for directories (and junctions, which are also directories), therefore this failure may just mean that "path" refers to a directory.
Then I think we can call `GetFileAttributesW` first to check if the `path` refers to a reparse point or not. If it does, we can just use `FILE_FLAG_OPEN_REPARSE_POINT`, otherwise, we skip and return the original path. WDYT?
And we can tell from the attribute if the `path` is a directory or not, if it is, we set `FILE_FLAG_BACKUP_SEMANTICS`
Yes, I think calling GetFileAttributeW then CreateFileW (with FILE_ATTRIBUTE_NORMAL or FILE_FLAG_BACKUP_SEMANTICS, depending on the attributes) is as good as calling CreateFileW first with FILE_FLAG_BACKUP_SEMANTICS then with FILE_ATTRIBUTE_NORMAL -- both approaches achieve the same goal and both are susceptible to file system races (i.e. when the file is modified between the two function calls).
OK, it's done. But I think we don't need `FILE_ATTRIBUTE_NORMAL` for file symlink because in the document it says `This attribute is valid only if used alone.`
https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea

Patch that resolved the issue:
diff --git a/src/main/cpp/util/file_platform.h b/src/main/cpp/util/file_platform.h
index 76e94d175732ed..868d3c90f21d82 100644
--- a/src/main/cpp/util/file_platform.h
+++ b/src/main/cpp/util/file_platform.h
@@ -209,6 +209,16 @@ void ForEachDirectoryEntry(const std::string &path,
 std::wstring GetCwdW();
 bool MakeDirectoriesW(const std::wstring &path, unsigned int mode);
 
+// Resolve a symlink to its target.
+// If `path` is a symlink, result will contain the target it points to,
+// If `path` is a not a symlink, result will contain `path` itself.
+// If the resolving succeeds, this function returns true,
+// otherwise it returns false.
+bool ReadSymlinkW(const std::wstring& path, std::wstring* result);
+
+// Check if `path` is a directory.
+bool IsDirectoryW(const std::wstring& path);
+
 // Interface to be implemented by ForEachDirectoryEntryW clients.
 class DirectoryEntryConsumerW {
  public:
diff --git a/src/main/cpp/util/file_windows.cc b/src/main/cpp/util/file_windows.cc
index d266f960c28ac0..ecc8c02026d4eb 100644
--- a/src/main/cpp/util/file_windows.cc
+++ b/src/main/cpp/util/file_windows.cc
@@ -46,7 +46,7 @@ using std::wstring;
 // Returns true if `path` refers to a directory or (non-dangling) junction.
 // `path` must be a normalized Windows path, with UNC prefix (and absolute) if
 // necessary.
-static bool IsDirectoryW(const wstring& path);
+bool IsDirectoryW(const wstring& path);
 
 // Returns true the file or junction at `path` is successfully deleted.
 // Returns false otherwise, or if `path` doesn't exist or is a directory.
@@ -549,6 +549,100 @@ bool JunctionResolver::Resolve(const WCHAR* path, unique_ptr<WCHAR[]>* result) {
   return Resolve(path, result, kMaximumJunctionDepth);
 }
 
+class SymlinkResolver {
+ public:
+  SymlinkResolver();
+
+  // Resolves symlink to its actual path.
+  //
+  // Returns true if `path` is not a symlink and it exists.
+  // Returns true if `path` is a symlink and can be successfully resolved.
+  // Returns false otherwise.
+  //
+  // If `result` is not nullptr and the method returned true, then this will be
+  // reset to point to a new WCHAR buffer containing the resolved path.
+  // If `path` is a symlink, this will be the resolved path, otherwise
+  // it will be a copy of `path`.
+  bool Resolve(const WCHAR* path, std::unique_ptr<WCHAR[]>* result);
+
+ private:
+  // Symbolic Link Reparse Data Buffer is described at:
+  // https://msdn.microsoft.com/en-us/library/cc232006.aspx
+  typedef struct _ReparseSymbolicLinkData {
+    static const int kSize = MAXIMUM_REPARSE_DATA_BUFFER_SIZE;
+    ULONG ReparseTag;
+    USHORT ReparseDataLength;
+    USHORT Reserved;
+    USHORT SubstituteNameOffset;
+    USHORT SubstituteNameLength;
+    USHORT PrintNameOffset;
+    USHORT PrintNameLength;
+    ULONG Flags;
+    WCHAR PathBuffer[1];
+  } ReparseSymbolicLinkData;
+
+  uint8_t reparse_buffer_bytes_[ReparseSymbolicLinkData::kSize];
+  ReparseSymbolicLinkData* reparse_buffer_;
+};
+
+SymlinkResolver::SymlinkResolver()
+    : reparse_buffer_(
+          reinterpret_cast<ReparseSymbolicLinkData*>(reparse_buffer_bytes_)) {
+}
+
+bool SymlinkResolver::Resolve(const WCHAR* path, unique_ptr<WCHAR[]>* result) {
+  DWORD attributes = ::GetFileAttributesW(path);
+  if (attributes == INVALID_FILE_ATTRIBUTES) {
+    // `path` does not exist.
+    return false;
+  } else {
+    if ((attributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0) {
+      bool is_dir = attributes & FILE_ATTRIBUTE_DIRECTORY;
+      AutoHandle handle(CreateFileW(path,
+                                    FILE_READ_EA,
+                                    FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
+                                    NULL,
+                                    OPEN_EXISTING,
+                                    (is_dir ? FILE_FLAG_BACKUP_SEMANTICS : 0)
+                                    | FILE_FLAG_OPEN_REPARSE_POINT,
+                                    NULL));
+      if (!handle.IsValid()) {
+        // Opening the symlink failed for whatever reason. For all intents and
+        // purposes we can treat this file as if it didn't exist.
+        return false;
+      }
+      // Read out the reparse point data.
+      DWORD bytes_returned;
+      BOOL ok = ::DeviceIoControl(
+          handle, FSCTL_GET_REPARSE_POINT, NULL, 0, reparse_buffer_,
+          MAXIMUM_REPARSE_DATA_BUFFER_SIZE, &bytes_returned, NULL);
+      if (!ok) {
+        // Reading the symlink data failed. For all intents and purposes we can
+        // treat this file as if it didn't exist.
+        return false;
+      }
+      if (reparse_buffer_->ReparseTag == IO_REPARSE_TAG_SYMLINK) {
+        if (result) {
+          size_t len = reparse_buffer_->SubstituteNameLength / sizeof(WCHAR);
+          result->reset(new WCHAR[len + 1]);
+          const WCHAR * substituteName = reparse_buffer_->PathBuffer
+                                         + (reparse_buffer_->SubstituteNameOffset / sizeof(WCHAR));
+          wcsncpy_s(result->get(), len + 1, substituteName, len);
+          result->get()[len] = UNICODE_NULL;
+        }
+        return true;
+      }
+    }
+  }
+  // `path` is a normal file or directory.
+  if (result) {
+    size_t len = wcslen(path) + 1;
+    result->reset(new WCHAR[len]);
+    memcpy(result->get(), path, len * sizeof(WCHAR));
+  }
+  return true;
+}
+
 bool ReadDirectorySymlink(const string& name, string* result) {
   wstring wname;
   string error;
@@ -566,6 +660,23 @@ bool ReadDirectorySymlink(const string& name, string* result) {
   return true;
 }
 
+bool ReadSymlinkW(const wstring& name, wstring* result) {
+  wstring wname;
+  string error;
+  if (!AsAbsoluteWindowsPath(name, &wname, &error)) {
+    BAZEL_DIE(blaze_exit_code::LOCAL_ENVIRONMENTAL_ERROR)
+        << "ReadSymlinkW(" << name
+        << "): AsAbsoluteWindowsPath failed: " << error;
+    return false;
+  }
+  unique_ptr<WCHAR[]> result_ptr;
+  if (!SymlinkResolver().Resolve(wname.c_str(), &result_ptr)) {
+    return false;
+  }
+  *result = RemoveUncPrefixMaybe(result_ptr.get());
+  return true;
+}
+
 bool PathExists(const string& path) {
   if (path.empty()) {
     return false;
@@ -776,7 +887,7 @@ bool CanAccessDirectory(const std::string& path) {
   return true;
 }
 
-static bool IsDirectoryW(const wstring& path) {
+bool IsDirectoryW(const wstring& path) {
   DWORD attrs = ::GetFileAttributesW(path.c_str());
   return (attrs != INVALID_FILE_ATTRIBUTES) &&
          (attrs & FILE_ATTRIBUTE_DIRECTORY) &&


Test patch: