Dynamics CRM / Any ASP.NET App - Improve C# dynamic compilation times
Published: 2023-04-19 08:14:41

# The Pain
Since I installed Dynamics 365 (CRM) v9.1 On-Premises, I get REALLY slow compile times. The **csc.exe (C# Compiler) is so slow** to compile whenever I make **unique request**, I could wait for many seconds or minutes for compilation to finish. So I was searching for a way to improve things. It doesn't do parallel compilation if I make multiple unique requests from multiple browser tabs and it doesn't utilize CPU that much. A "[legacy junk](https://stackoverflow.com/a/70342126/50173)" indeed.

The ASP.net does dynamic compilation on first request. You can read more about it at [Understanding ASP.NET Dynamic Compilation.](https://learn.microsoft.com/en-us/previous-versions/aspnet/ms366723(v=vs.100)). But from Task Manager it looks something like this: 
```
"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\csc.exe" /noconfig /fullpaths @"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\0345f868\a571ab7d\mejffpag.cmdline"
```

# The Painkillers?
As Dynamics 365 is a 3rd party app and I thought I could at least precompile the aspx pages and such. So I did just that:

```
. C:\windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_compiler.exe -m "/LM/W3SVC/2/ROOT"
```

As it did take a while to complete, I certainly had good feelings about this. Just to find out csc.exe still doing dynamic compilation when I make requests and the performance for my dev environment hasn't improved at all (just a gut feeling, didn't do measurements). That means I have to take first request hit all of my own for any request. :(

But it did something, as it generated tons of files within `Temporary ASP.NET Files` folder - probably just not enought.

# The Painkillers!
After a while I stumbled upon something interesting: [Enabling the .NET Compiler Platform (“Roslyn”) in ASP.NET applications](https://devblogs.microsoft.com/dotnet/enabling-the-net-compiler-platform-roslyn-in-asp-net-applications/).

Eager to try out and do some not-so-much-supported modifications within 3rd party app web.config that will certainly be overwritten when I apply next patch - but it is a dev env, no real risk of breaking it:

```powershell
# Got nuget.exe and within arbitrary directory:
nuget.exe install Microsoft.CodeDom.Providers.DotNetCompilerPlatform -Version 3.6.0 -OutputDirectory .\packages\

# note that Dynamics 365 (CRM) 9.1 Is .NET Framework 4.6.2 app.
cp -Recurse ".\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.3.6.0\tools\Roslyn46\*" "C:\Program Files\Dynamics 365\CRMWeb\bin\roslyn"
cp ".\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.3.6.0\lib\net45\*" "C:\Program Files\Dynamics 365\CRMWeb\bin\"
```

Then edited "C:\Program Files\Dynamics 365\CRMWeb\web.config" by adding `<system.codedom>` node:
```xml
<configuration>
  ...
  <system.codedom>
    <compilers>
      <compiler language="c#;cs;csharp" extension=".cs"
        type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.CSharpCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=3.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
        warningLevel="4" compilerOptions="/langversion:7.3 /nowarn:1659;1699;1701"/>
      <compiler language="vb;vbs;visualbasic;vbscript" extension=".vb"
        type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=3.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
        warningLevel="4" compilerOptions="/nowarn:41008 /define:_MYTYPE=\&quot;Web\&quot; /optionInfer+"/>
    </compilers>
  </system.codedom>
</configuration>
```

I did validate that it was using the new compiler: `csc.exe` was from `bin/roslyn` directory and I saw a `VBCSCompiler.exe` process too. That verifies that web.config changes are correct and assemblies are in correct places.

`Restart-Service W3SVC` wasn't needed, as changes to web.config are detected and app pool gets recycled.

# Performance improvements
So I had to measure performance improvement. I opted for [UIforETW](https://github.com/google/UIforETW) to initiate hassle free etw trace and [Windows Performance Analyzer](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-analyzer) (WPA) to see the results. I wanted _more scientific_ results than a wall clock :)

- I did a run with roslyn compiler and other run with builtin compiler.
- I did open root Dynamics CRM page (a dashboard). I still use Legacy interface, but it doesn't prevent us making a point on csc.exe performance.
- Compilation resulted in generating 440 files.
- I did purge ASP.NET Temporary Files before each run.

## Legacy csc.exe

1. Purge files before run

```ps
gci -Recurse "C:\windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\0345f868\a571ab7d" | ? lastwritetime -ge '2023-04-18' | ? PsIsContainer -eq $false | Remove-Item -Force
```

2. Refresh page
3. Look at trace with WPA - 6 times csc.exe process was called. Time since first spawn and last one ending 199sec = **3m19sec**.

    ![WPA results for legacy csc.exe](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5titxugt5v0un1ij69ii.png)

## Roslynized csc.exe

1. Purge files before run
2. Refresh Page
3. **116sec = 1m56sec**
    
![WPA results for roslyn csc.exe](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gkauoc5hupv0gg9bjm0v.png)

## Results

Okay, so that was kind of total refresh for a single page that makes many calls - normally things would be somewhat cached. I feel that navigating to other pages feels faster, but still I wouldn't say that it is fast.

I'll leave roslyn compiler on as any performance improvement will help - I feel that normally navigating through pages is faster. It helps against doing context switching to other tasks while waiting on compilation.

# Precompiling ASP.NET Framework app 
We covered dynamic compilation - a process where compilation for relevant files happens when making request and compiling appropriate files - and that request is taking the performance impact.

For ASP.NET apps you develop, you can [precompile your website](https://learn.microsoft.com/en-us/aspnet/web-forms/overview/older-versions-getting-started/deploying-web-site-projects/precompiling-your-website-cs), so that dynamic compilation never takes place and your users don't have to wait.

I also have access to a ASP.NET Framework 4.8 application and I did want to improve build times there too. It usually took 34 minutes for aspnet_compiler.exe to do its precompilation job. All I had to do was to install `Microsoft.CodeDom.Providers.DotNetCompilerPlatform` package (latest 4.1.0 version), it did the web.config changes and copied required files to bin directory - the result was immediatelly visible - instead of 34 minutes, now it did precompilation in ~13 minutes. Solid 20 minutes shaved off!

As for precompilation, it wasn't immediatelly obvjous that it is using roslyn csc.exe - because I didn't see neither `VBCSCompiler.exe` or `csc.exe`, just `aspnet_compiler.exe` doing its job. But the results speak of themselves.

This topic was very helpful in getting up and running with roslyn-ized csc.exe compiler: [How to speed up aspnet_compiler.exe?](https://stackoverflow.com/questions/70103543/how-to-speed-up-aspnet-compiler-exe)