Repository: TypeScript
Issue: Support per volume case sensitivity
Labels: Suggestion,Awaiting More Feedback

### 🔎 Search Terms

"tsserver", "cache invalidation", "new file", "case sensitive", "IntelliSense", "inferred project", "mac os", "watch", "tsserver restart"

### 🕗 Version & Regression Information

This bug is reproducible between TypeScript 5.1.6 and 5.3.0-nightly (commit b5557271a51704e0469dd86974335a4775a68ffd).

I can test versions prior to 5.1.6 if necessary.

### ⏯ Playground Link

_This bug is specific to a local editing environment and file system quirks. A playground would not be able reproduce this._

### 🪜 Steps to Reproduce

A few developers on my team have noticed that IntelliSense operations intermittently do not work in newly created files in VS Code. IntelliSense works in the new file after restarting TSServer.

Breakpoint debugging TSServer, I believe there's a file system caching bug related to case sensitivity. The bug can be reproduced consistently on macOS.

1. Open a directory in VS Code with a `tsconfig.json` file on a case-sensitive volume.
3. Ensure the TypeScript version is set to "Use VS Code's Version". The problem only reproduces when `tsserver.js` is on a case-**in**sensitive volume, which is usually true for `/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/node_modules/typescript/lib/tsserver.js`.
4. Create a new file with at least one capital letter in its name. (e.g. `Foo.tsx`).
5. Observe that IntelliSense operations performed within the new `Foo.tsx` file do not have information on other files of the same `tsconfig.json` program.

### 🙁 Actual behavior

I've created a small repo to help reproduce the bug. There isn't anything special about the repo. It's more where the repo is cloned to and the file system case sensitivity that matters. https://github.com/gluxon/tsserver-new-file-intellisense-bug

Here's a video of the bug.

1. The newly created `bar.ts` file is able to see IntelliSense information from the rest of the project.
2. The newly created `Baz.ts` file is not.

https://github.com/microsoft/TypeScript/assets/906558/c43850c1-7b99-40a3-a68a-b2ae8d3d5388

Looking over the TSServer logs for the video above, we can see that `bar.ts` is matched with the right project. The `Baz.ts` file is instead mapped to `/dev/null/inferredProject2*`.

```log
Info 743  [15:52:29.973] Open files: 
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/foo.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /Volumes/Code/tsserver-new-file-intellisense-bug/tsconfig.json
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/bar.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /Volumes/Code/tsserver-new-file-intellisense-bug/tsconfig.json
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /dev/null/inferredProject2*
```

### 🙂 Expected behavior

TSServer always associates new files with the `tsconfig.json` file that includes it instead of an inferred project.

---

## 🔬 Self-Debugging

We noticed that this problem does not reproduce when `tsserver.js` is loaded from the same case-sensitive file system. It only happens when `tsserver.js` is loaded from a case-**in**sensitive file system. The main way of changing where `tsserver.js` was loaded from was through `typescript.tsdk` in `.vscode/settings.json`.

![Screenshot 2023-08-17 at 7 13 28 PM](https://github.com/microsoft/TypeScript/assets/906558/f70f1c0e-4b1c-4579-8195-2e901daff666)

With that toggle, we were able to reliably control whether or not the problem reproduces and compare TSServer's internal state for the successful and failing cases. Skip to the bottom for the conclusion.

### Program Loading and Caching

Starting at a high-level, we've confirmed in a debugger a few important pieces of information:

1. The TypeScript [`Program`](https://github.com/microsoft/typescript/blob/704e7661e231d32734964ae82682640a710cc7e3/src/compiler/program.ts#L1482) object's `rootFilesMap` records what files are part of a `tsconfig.json`. The new file is not added to the expected program's `rootFilesMap` in the failing case, but is added in the successful case. It makes sense why TSServer associates the newly created file to an inferred project given this.
2. The `rootFiles` and `rootFilesMap` fields are computed through [`getFileNamesFromConfigSpecs(...)`](https://github.com/microsoft/TypeScript/blob/b29ed184664299741573ad5266940accde690042/src/compiler/commandLineParser.ts#L3585). This function does not return the newly created file in the failing case (but does in the successful case).
3. The `getFileNamesFromConfigSpecs` function calls `host.readDirectory(...)`. This `readDirectory` call does not return the newly created file in the failing case, but does in the successful case.

The `host` object in `getFileNamesFromConfigSpecs` is an instance of the [`CachedDirectoryStructureHost`](https://github.com/microsoft/typescript/blob/913f65c28da5d4b79180d1a65d1f596f56ae012a/src/compiler/watchUtilities.ts#L113)

### Debugging `CachedDirectoryStructureHost`.

The `CachedDirectoryStructureHost` object is kept up to date through `addOrDeleteFileOrDirectory` in `src/compiler/watchUtilities.ts`. We believe the fundamental difference between the failing and successful cases is the evaluation of this line

https://github.com/microsoft/TypeScript/blob/913f65c28da5d4b79180d1a65d1f596f56ae012a/src/compiler/watchUtilities.ts#L315

#### Failing Case

- The `fileOrDirectoryPath` const evaluates to `/volumes/code/tsserver-new-file-intellisense-bug/baz.ts`
- `fsQueryResult.fileExists` subsequently becomes `false`.

![Screenshot 2023-08-17 at 7 31 25 PM](https://github.com/microsoft/TypeScript/assets/906558/29304860-fff7-43b7-ae0a-d6fcedf595bd)

Note that the entire `fileOrDirectoryPath` string has been been shifted to be lowercase.

#### Successful Case

- The `fileOrDirectoryPath` const evaluates to `/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts`
- `fsQueryResult.fileExists` subsequently becomes `true`.

![Screenshot 2023-08-17 at 7 33 41 PM](https://github.com/microsoft/TypeScript/assets/906558/fb1bd716-c827-494d-8bbb-db7d0ac2e411)

In this scenario, the casing of the new file is preserved.

### Why is `fsQueryResult.fileExists` important?

The value of `fsQueryResult.fileExists` is passed to `updateFilesOfFileSystemEntry`.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/watchUtilities.ts#L324

Once `updateFilesOfFileSystemEntry` runs, this changes whether or not the `CachedDirectoryStructureHost` sees the newly created file. I'm able to verify this with a breakpoint on this line, and evaluating the results of `readDirectory` in the debug console.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/watchUtilities.ts#L326

In the successful case

```ts
❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
false

// ...updateFilesOfFileSystemEntry(...)

❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
true
```

In the failing case, this always evaluates to `false`, even after the `updateFilesOfFileSystemEntry` call.

```ts
❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
false
```

Ultimately, I think the problem is before the call to `updateFilesOfFileSystemEntry` problem though. I would guess `updateFilesOfFileSystemEntry` should be passed a value of `fsQueryResult.fileExists` that's `true` regardless of file system case sensitivity.

### How does TypeScript decide when to lowercase?

The `fileOrDirectoryPath` may or may not be lowercased through:

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1505

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1066-L1068

The `this.toCanonicalFileName` function performs lower-casing based on `this.host.useCaseSensitiveFileNames`.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/core.ts#L2604

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1028

There's likely an assumption to unwind related to `this.host.useCaseSensitiveFileNames`. I'm not sure if this is the TypeScript team's preferred fix, but I suspect we need to compute a different `toCanonicalFileName` function per-project or per-watched folder since the case sensitivity of an opened project folder is independent to where `tsserver.js` loads from.

### Conclusion

The `CachedDirectoryStructureHost` has a bug where it does not update its `cachedReadDirectoryResult` internal variable if (1) newly created files have a capital letter and (2) `tsserver.js` is spawned from a file system volume that's case-insensitive.