Repository: node
Issue: Node.js directory case issue on Windows with AppV enabled
Labels: fs,windows

### Version

16.x and previous LTS versions of Node.js

### Platform

Microsoft Windows NT 10.0.14393.0 x64 (Windows Server 2016)

### Subsystem

_No response_

### What steps will reproduce the bug?

This issue is reproducible on **Windows** machines, with **Microsoft AppV** running, and where the `AppvStrm` filter driver is **enabled**. This has been reproduced on a dozen machines meeting these conditions.

1. Create a file named "mkdirs-test.js" with the below code:
2. Execute `node mkdirs-test.js`
3. If your system is affected, then you will see uppercase directories printed to standard out.

```javascript
// mkdirs-test.js
const fs = require('fs')
const {promisify} = require('util')
const mkdirp = promisify(fs.mkdir)

let paths = [
  "aaa",
  "aaa/bbb",
  "aaa/bbb/ccc",
  "aaa/bbb/ccc/ddd",
  "aaa/bbb/ccc/ddd/eee",
]

let round = 1;

function runTest() {
  console.log("round " + round++)
  cleanup()

  let promises = []
  for (let i = 0; i < paths.length; i++) {
    // create folder, ignore error - if parent folder is not created yet, mkdir will just fail
    promises.push(mkdirp(paths[i]).catch(_ => null))

    // adding a nominal amount of sleep between mkdirs seem to reduce the frequency of uppercase folders
    //promises.push(sleep(Math.floor(Math.random() * 10) + 1).then(() => mkdirp(paths[i]).catch(_ => null)))
  }

  return Promise.all(promises).then(() => {
    analyze()
  })
}

function cleanup() {
  if (fs.existsSync("aaa")) {
    if (fs.rmSync) {
      fs.rmSync("aaa", {recursive: true})
    } else {
      fs.rmdirSync("aaa", {recursive: true})
    }
  }
}

function sleep(millis) {
  return new Promise(resolve => setTimeout(resolve, millis));
}

function repeat(func, times) {
  let promise = Promise.resolve();
  while (times-- > 0) promise = promise.then(func);
  return promise;
}

repeat(runTest, 100).then(function () {
  console.log("done");
});

function analyze() {
  paths.forEach(p => {
    if (fs.existsSync(p)) {
      let realpath = fs.realpathSync.native(p).replace(fs.realpathSync.native(process.cwd()) + "\\", "")
      let folder = realpath.split("\\").pop()
      if (folder.toLowerCase() !== folder) {
        console.log("  upper => " + realpath)
      }
    }
  })
}
```

#### What this script does

This script is based on code used by npm 7 and higher (npm 8 is the current version bundled with Node.js 16 LTS) that is used to install dependency packages. Source: https://github.com/npm/arborist/blob/main/lib/arborist/reify.js#L492-L517

Specifically, this script simulates the way directories are created by npm 7 and higher when performing an `npm install`. The script uses `fs.mkdir` wrapped in a promise to create a nest of directories in the current working directory. So after executing you should have a directory structure matching `<cwd>/aaa/bbb/ccc/ddd/eee`. The script will do this 100 "rounds" which simulates install of many child dependencies. Also note that many concurrent, async executions of `fs.mkdir` appears to play a key part in causing this issue; creating only a single set of nested directories is not always sufficient to reproduce the problem, hence why this script executes 100 rounds.

### How often does it reproduce? Is there a required condition?

This issue is reproducible on **Windows** machines, with **Microsoft AppV** running, and specifically have the `AppvStrm` filter driver **enabled**. On affected machines, this issue is reproducible nearly every time the test script is run or `npm install` is executed using npm 7 or higher (please see Additional Details section for more on this).

### What is the expected behavior?

The expected behavior is directories are created with the requested lowercase. 

The output from the test script should be the following, which indicates no uppercase directories were created. This expected output is possible if the `AppvStrm` filter driver is disabled on an affected machine.

```
// expected output of mkdirs-test.js
round 1
round 2
round 3
round 4
round 5
round 6
round 7
round 8
round 9
round 10
round 11
round 12
round 13
round 14
round 15
round 16
round 17
round 18
round 19
round 20
round 21
round 22
round 23
round 24
round 25
round 26
round 27
round 28
round 29
round 30
round 31
round 32
round 33
round 34
round 35
round 36
round 37
round 38
round 39
round 40
round 41
round 42
round 43
round 44
round 45
round 46
round 47
round 48
round 49
round 50
round 51
round 52
round 53
round 54
round 55
round 56
round 57
round 58
round 59
round 60
round 61
round 62
round 63
round 64
round 65
round 66
round 67
round 68
round 69
round 70
round 71
round 72
round 73
round 74
round 75
round 76
round 77
round 78
round 79
round 80
round 81
round 82
round 83
round 84
round 85
round 86
round 87
round 88
round 89
round 90
round 91
round 92
round 93
round 94
round 95
round 96
round 97
round 98
round 99
round 100
done
```


### What do you see instead?

The actual behavior observed is that directories are created with an uppercase name, instead of lowercase specified in test script. 

The output from the script is the following. A round with 'upper' immediately following it means for that trial round a directory in the path was created with uppercase.

```
// actual output of mkdirs-test.js (when AppV filter is enabled)
round 1
round 2
  upper => aaa\bbb\CCC
round 3
round 4
  upper => aaa\bbb\ccc\DDD
round 5
  upper => aaa\bbb\CCC
round 6
  upper => aaa\bbb\ccc\DDD
round 7
round 8
round 9
round 10
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 11
round 12
  upper => aaa\bbb\ccc\DDD
round 13
  upper => aaa\bbb\CCC
round 14
round 15
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 16
  upper => aaa\bbb\CCC
round 17
  upper => aaa\bbb\ccc\DDD
round 18
  upper => aaa\bbb\ccc\DDD
round 19
  upper => aaa\bbb\CCC
round 20
round 21
  upper => aaa\bbb\CCC
round 22
round 23
round 24
  upper => aaa\bbb\ccc\DDD
round 25
round 26
  upper => aaa\bbb\CCC
round 27
  upper => aaa\bbb\ccc\DDD
round 28
round 29
round 30
round 31
round 32
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 33
round 34
  upper => aaa\bbb\ccc\DDD
round 35
  upper => aaa\bbb\ccc\DDD
round 36
  upper => aaa\bbb\ccc\DDD
round 37
  upper => aaa\bbb\ccc\DDD
round 38
round 39
round 40
round 41
  upper => aaa\bbb\CCC
round 42
round 43
  upper => aaa\bbb\CCC
round 44
round 45
round 46
round 47
  upper => aaa\bbb\ccc\DDD
round 48
  upper => aaa\bbb\ccc\DDD
round 49
  upper => aaa\bbb\ccc\DDD
round 50
round 51
round 52
round 53
  upper => aaa\bbb\CCC
round 54
round 55
  upper => aaa\bbb\ccc\DDD
round 56
  upper => aaa\bbb\CCC
round 57
  upper => aaa\bbb\ccc\DDD
round 58
  upper => aaa\bbb\ccc\DDD
round 59
round 60
round 61
round 62
round 63
  upper => aaa\bbb\ccc\DDD
round 64
  upper => aaa\bbb\CCC
round 65
  upper => aaa\bbb\ccc\DDD
round 66
round 67
  upper => aaa\bbb\ccc\DDD
round 68
round 69
round 70
round 71
  upper => aaa\bbb\ccc\DDD
round 72
round 73
round 74
  upper => aaa\bbb\ccc\DDD
round 75
round 76
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 77
  upper => aaa\bbb\CCC
round 78
  upper => aaa\bbb\ccc\DDD
round 79
round 80
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 81
round 82
  upper => aaa\bbb\ccc\DDD
round 83
  upper => aaa\bbb\ccc\DDD
round 84
round 85
round 86
  upper => aaa\bbb\CCC
round 87
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 88
round 89
  upper => aaa\bbb\CCC
round 90
  upper => aaa\bbb\ccc\DDD
round 91
  upper => aaa\bbb\ccc\DDD
round 92
  upper => aaa\bbb\ccc\DDD
round 93
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 94
  upper => aaa\bbb\CCC
  upper => aaa\bbb\CCC\DDD
round 95
  upper => aaa\bbb\ccc\DDD
round 96
  upper => aaa\bbb\CCC
round 97
round 98
  upper => aaa\bbb\ccc\DDD
round 99
round 100
done
```

### Additional information

First, thank you Node team and community for reading this issue so far! This has been a bizarre issue that I have been trying to track down for a while now. Appreciate the consideration and help.

#### What is the impact of this issue?

**This issue is preventing devs at my company from upgrading to Node.js 16 and npm 7 or higher.** We are continuing to use the Node.js 14 release because it comes with npm 6 which does not exhibit this issue.

#### Background

I am a dev in an enterprise which uses Windows machines and where Microsoft AppV is installed and used to deliver certain applications. Note, however, that Node.js is NOT being delivered via AppV in my case; the Node.js binary is downloaded from Nodejs.org following standard instructions. AppV just happens to be on the machine and enables certain "filter drivers" which appear to be interfering with Node fs.mkdir.

#### How this issue was discovered

This issue was first discovered when I tried to upgrade our applications from npm 6 to npm 8 (or npm 7). Using npm 8, a fresh `npm install` (no project node_modules or package-lock) would result in a variation of the error:

```
npm ERR! enoent ENOENT: Cannot cd into 'C:/Users/myname/projects/sample-npm8-app/node_modules/eslint-plugin-testing-library/node_modules/eslint-visitor-keys'
```

Below is a package.json with a single dependency that reproduces the original issue. Note, this is not specific to react-scripts and can be reproduced many different sets of dependencies. The common requirement for failure seems to be installing a package.json that will pull-in a large number of transient dependencies, thereby creating a large number of child directories under node_modules. 

```json
{
  "name": "sample",
  "version": "0.1.0",
  "private": true,
  "devDependencies": {
    "react-scripts": "~4.0.3"
  }
}
```

Running `npm install` results in failure nearly every time and the directory mentioned in the error message is not consistent - it may be different each time. 

Some other dependencies that will reproduce this failure:
- `@craco/craco`
- `@testing-library/react-hooks`

Looking into node_modules I notice some directories were getting created with uppercase letters - this explains why the process couldn't find the expected directory, but why was this happening? I started digging into npm's source code and found starting with npm 7 the i/o code had significantly changed from npm 6. I found it was calling `fs.mkdir` async to create directories. That led me to writing the very simple test script attached to this issue to reproduce. 

I cannot reproduce this on any personal machines only on my company's machines using Windows and AppV. The issue goes away when disabling one of many filters that AppV enables; the name of this filter is `AppvStrm`. I am not clear what this filter does, however disabling it is not a straight forward fix because it appears to be needed by other software. Since this is an enterprise, I don't have admin access to my machine and it is not possible to simply ask our admins to disable this filter for devs. I am hoping there is an expert out there that has come across this issue and may be able to shed some light on why this is happening. We have started engaging Microsoft support, but so far have not made much progress.