Repository: TypeScript
Issue: Do not perform typechecking if files are unchanged when compiling with `-p`
Labels: Suggestion,In Discussion,Domain: tsc -b,Domain: --incremental

## Search Terms

incremental, composite

# Chrome DevTools and TypeScript

TLDR: integrate/improve incremental build functionality into `-p`
 
**Please see the summary at the bottom for the actual feature request in this issue. The rest of it is (important) background information as to why we are making this feature request.**
 
As you might be aware, [Chrome DevTools](https://chromium.googlesource.com/devtools/devtools-frontend/) is [migrating](http://crbug.com/1011811) from the Closure Compiler to the TypeScript compiler.
As part of the integration of TypeScript with [GN/Ninja](https://gn.googlesource.com/gn/+/master/), we have written a [desugaring Python script](https://source.chromium.org/chromium/chromium/src/+/master:third_party/devtools-frontend/src/third_party/typescript/ts_library.py;l=62-68;drc=4e556fa8176bdfb0dcc3845a7a07aa5e8d724466) to eventually call `tsc`.
 
The high-level process is as follows:
1. GN/Ninja figures out which GN actions need to run, based on the files that have been changed
1. One of these GN actions could be calling `ts_library.py`
1. The Python script first generates a `tsconfig.json`, based on its file inputs and general configuration. This `tsconfig.json` file is written to the filesystem, see below for an example
1. We call `tsc` with pinned versions of both Node and TypeScript and point it to the `tsconfig.json` file with the `-p` compiler flag
 
This setup is similar to `tsc -b`.
However, since Chrome DevTools is part of the Chromium codebase, we have to integrate with GN/Ninja.
As such, GN/Ninja is "running the world", rather than a tool like TypeScript.
Therefore, we are not able to use `tsc -b`, as it assumes that `tsc` is the tool "running the world".
 
In general, this setup works.
Sadly, one area that we do have some issues is related to the performance of the TypeScript compiler.
 
## Performance investigation
 
There are two areas of interest for our integration: the performance of both a clean and an incremental build.
 
For a clean build, we are mostly bound by the performance of the TypeScript compiler itself.
Since we have no prior information, we can only take advantage of compiler options that improve performance.
For example, we have been using `--skipLibCheck` for all targets except one, as we can assume that libs generally don't have problems across multiple different subfolders.
 
For an incremental build, the situation is a bit different.
Since GN/Ninja is quite smart at figuring out when (not) to run a GN action, we have optimized our TypeScript integration to only run if strictly necessary.
To do so, we are taking advantage of `.tsbuildinfo` files and general caching of results.
 
Sadly, even for incremental builds we are observing quite long compilation times.
Therefore, I decided to do a performance investigation in the TypeScript compiler explicitly for its incremental build performance.
 
## Incremental build analysis
 
The base assumption that I operated on was the following:
 
> Given two consecutive invocations of `tsc` without any file changes, the second `tsc` invocation should perform minimal (if at all any) work
 
However, I quickly realized that this assumption is not the case.
The scenario that I tested was the following:
 
1. Given that I have performed a fresh build of DevTools
1. Verify that a rebuild with GN/Ninja shows "no work to do"
1. Call `tsc` manually **as if it were part of a normal GN action** and observe its performance
 
The command I used to analyze its performance was the following:
 
```
$ time third_party/node/node.py --output --trace-ic node_modules/typescript/lib/tsc.js -p out/Default/gen/front_end/sdk/sdk-tsconfig.json --extendedDiagnostics --generateCpuProfile profile.cpuprofile
```
 
Example output (collapsed for brevity):
 
<details>
 
```
$ time third_party/node/node.py --output node_modules/typescript/lib/tsc.js -p out/Default/gen/front_end/sdk/sdk-tsconfig.json --extendedDiagnostics
Files:                         170
Lines:                       87519
Nodes:                      322413
Identifiers:                112717
Symbols:                     83238
Types:                       22788
Instantiations:              24303
Memory used:               148140K
Assignability cache size:     3899
Identity cache size:          1478
Subtype cache size:            597
Strict subtype cache size:     499
I/O Read time:               0.02s
Parse time:                  0.98s
ResolveTypeReference time:   0.00s
ResolveModule time:          0.06s
Program time:                1.14s
Bind time:                   0.53s
Check time:                  2.54s
transformTime time:          0.91s
Total time:                  4.20s
 
real    0m5.524s
user    0m10.460s
sys     0m0.342s
```
 
</details>
 
Since DevTools has a lot of files/LoC, the summation of the invocation times adds up to minutes. In this analysis, I chose the `sdk` folder, as Ninja reports that it is the slowest part of the DevTools build (log collapsed for brevity):
 
<details>
 
```
$ NINJA_SUMMARIZE_BUILD=1 autoninja -C out/Release -w dupbuild=err
depot_tools/ninja -C out/Release -w dupbuild=err -j 10 -d stats
ninja: Entering directory `out/Release'
[1 processes, 1/1 @ 3.2/s : 0.312s ] Regenerating ninja files
[1 processes, 1502/1502 @ 4.7/s : 322.149s ] STAMP obj/generate_devtools_grd.stamp
metric                count   avg (us)  total (ms)
.ninja parse          4       48223.8   192.9
canonicalize str      50764   0.2       7.8
canonicalize path     51274   0.1       4.3
lookup node           57528   0.2       9.8
.ninja_log load       2       14999.0   30.0
.ninja_log recompact  1       322624.0  322.6
node stat             24605   17.1      421.2
.ninja_deps load      2       175.5     0.4
depfile load          2       435.0     0.9
StartEdge             1504    1378.1    2072.7
FinishCommand         1503    149.8     225.2
 
path->node hash load 0.78 (9599 entries / 12289 buckets)
   Longest build steps:
          2.3 weighted s to build (38 items) gen/front_end/perf_ui/perf_ui-tsconfig.json, gen/front_end/perf_ui/perf_ui-tsconfig.json.tsbuildinfo, ... (13.2 s elapsed time)
          2.3 weighted s to build (29 items) gen/front_end/console/console-tsconfig.json, gen/front_end/console/console-tsconfig.json.tsbuildinfo, ... (13.7 s elapsed time)
          3.2 weighted s to build (65 items) gen/front_end/profiler/profiler-tsconfig.json, gen/front_end/profiler/profiler-tsconfig.json.tsbuildinfo, ... (18.0 s elapsed time)
          3.3 weighted s to build (77 items) gen/front_end/network/network-tsconfig.json, gen/front_end/network/network-tsconfig.json.tsbuildinfo, ... (18.2 s elapsed time)
          3.5 weighted s to build (104 items) gen/front_end/sources/sources-tsconfig.json, gen/front_end/sources/sources-tsconfig.json.tsbuildinfo, ... (18.3 s elapsed time)
          3.5 weighted s to build (62 items) gen/front_end/resources/resources-tsconfig.json, gen/front_end/resources/resources-tsconfig.json.tsbuildinfo, ... (16.7 s elapsed time)
          3.6 weighted s to build (119 items) gen/front_end/elements/elements-tsconfig.json, gen/front_end/elements/elements-tsconfig.json.tsbuildinfo, ... (20.2 s elapsed time)
          3.7 weighted s to build (65 items) gen/front_end/timeline/timeline-tsconfig.json, gen/front_end/timeline/timeline-tsconfig.json.tsbuildinfo, ... (17.4 s elapsed time)
          3.8 weighted s to build (179 items) gen/front_end/ui/ui-tsconfig.json, gen/front_end/ui/ui-tsconfig.json.tsbuildinfo, ... (14.4 s elapsed time)
          4.7 weighted s to build (191 items) gen/front_end/sdk/sdk-tsconfig.json, gen/front_end/sdk/sdk-tsconfig.json.tsbuildinfo, ... (13.1 s elapsed time)
   Time by build-step type:
          0.1 s weighted time to generate 7 .css files (0.7 s elapsed time sum)
          0.2 s weighted time to generate 6 .html files (1.5 s elapsed time sum)
          0.4 s weighted time to generate 1 .grd files (0.4 s elapsed time sum)
          1.2 s weighted time to generate 800 .stamp files (7.5 s elapsed time sum)
          2.4 s weighted time to generate 84 .prebundle.ts files (14.3 s elapsed time sum)
          2.8 s weighted time to generate 95 .json files (16.2 s elapsed time sum)
         44.5 s weighted time to generate 187 .js files (272.7 s elapsed time sum)
        270.6 s weighted time to generate 322 .d.ts files (1811.4 s elapsed time sum)
   322.1 s weighted time (2124.7 s elapsed time sum, 6.6x parallelism)
   1502 build steps completed, average of 4.66/s
```
 
</details>
 
After analyzing the flamecharts produced by `tsc`, I observed that TypeScript was indeed checking the source files, even though technically no files had changed.
Yet in its flamechart, I found references to the incremental build, which we have turned on via `--composite` (which in turn implies `--incremental`).
 
The callstack included:
1. performIncrementalCompilation
1. createIncrementalProgram
1. createIncrementalCompilerHost
1. changeCompilerHostLikeToUseCache
 
Based on these functions, I ventured further and eventually found references to a function called `tryReuseStructureFromOldProgram`.
This function sounded very interesting, so I decided to figure out its callstack (`console.log(new Error().stack)`):
 
```js
Error
   at tryReuseStructureFromOldProgram (devtools-frontend/node_modules/typescript/lib/tsc.js:85780:25)
   at Object.createProgram (devtools-frontend/node_modules/typescript/lib/tsc.js:85464:30)
   at Object.getBuilderCreationParameters (devtools-frontend/node_modules/typescript/lib/tsc.js:88599:29)
   at createEmitAndSemanticDiagnosticsBuilderProgram (devtools-frontend/node_modules/typescript/lib/tsc.js:88876:107)
   at Object.createIncrementalProgram (devtools-frontend/node_modules/typescript/lib/tsc.js:90295:16)
   at Object.performIncrementalCompilation (devtools-frontend/node_modules/typescript/lib/tsc.js:90254:33)
   at performIncrementalCompilation (devtools-frontend/node_modules/typescript/lib/tsc.js:92484:29)
   at executeCommandLineWorker (devtools-frontend/node_modules/typescript/lib/tsc.js:92356:17)
   at devtools-frontend/node_modules/typescript/lib/tsc.js:92401:99
   at devtools-frontend/node_modules/typescript/lib/tsc.js:4422:25
```
 
`tryReuseStructureFromOldProgram` returns 0 (which implies its program could not be reused), as `oldProgram` does not exist.
 
However, when analyzing `createBuilderProgramState` I discovered that it was correctly deducing that there were no files changed.
`console.log(state.changedFilesSet);` logged an empty set.
This is correct, as no files had changed and the full program information from the `.tsbuildinfo` could be used.
 
At this point, I was a bit puzzled.
It seemed like `tsc` was able to figure out nothing had changed, yet it was still doing work.
Based on the content of the `.tsbuildinfo` file, I continued searching for its content.
There were two interesting fieldnames: `signature` and `version`.
 
When searching for `\.\bsignature\b`, I found two interesting functions:
 
`const computeHash = host.createHash || generateDjb2Hash;`.
 
Sadly `computeHash` is passed in as a method parameter into *a lot* of functions.
Therefore, it is difficult to figure out where it is actually used.
 
```ts
/**
* Returns if the shape of the signature has changed since last emit
*/
export function updateShapeSignature
```
 
The second function was a lot more interesting and also had references to `computeHash`.
Based on my reading of these functions, `tsc` can figure when (not) to compile a particular project.
This is (as expected) based on file hashes and checking (among other things) the compiler version it was previously compiled with.
 
While these functions seemed what I was looking for, adding logging to either of those showed that they were not called at all.
I added additional logging to numerous callsides of `updateShapeSignature`, yet none of these were called.
 
At this point, I was a bit confused as to how/why the `.tsbuildinfo` was seemingly used, but not used determining whether it should compile at all.
 
## TLDR: integrate/improve incremental build functionality into `-p`
 
Eventually I realized the following: `tsc -b` and `tsc -w` can make efficient decisions about recompilation.
These two modes can figure out whether recompilation is necessary and bail out if the above mentioned functions determine that nothing has changed.
 
However, `tsc -p` does not take advantage of this functionality.
To improve the incremental build performance of DevTools (where the assumption is that `tsc` is not "running the world"), can we extend `tsc -p` to prevent unnecessary checking when no files have changed?
 
Essentially, my expectation would be that the `time third_party/node/node.py` command I posted all the way at the top would do no (or near zero) work, if nothing has changed.
This would have significant performance improvements for DevTools, where a majority of the files rarely change and rebuilds are very frequent.

## Checklist

My suggestion meets these guidelines:

* [x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
* [x] This wouldn't change the runtime behavior of existing JavaScript code
* [x] This could be implemented without emitting different JS based on the types of the expressions
* [x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
* [x] This feature would agree with the rest of [TypeScript's Design Goals](https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals).