opened
ritchxu
username_0
In our particular case, we have a custom shell around `ConsoleShell` that upon start up loads `Microsoft.WindowsAzure.Storage.dll` into the current app domain. Once the shell is up, we couldn't import for example `Az.Accounts` module because it depends on a different version of `Microsoft.WindowsAzure.Storage.dll`. Since this is a more general problem with PowerShell Core, I posted the issue here instead of over at Azure PowerShell. Effectively, I'm looking for a workaround for issue https://github.com/PowerShell/PowerShell/issues/2083.

## Attempt 1: custom assembly resolve
```
using Microsoft.PowerShell;
using System;
using System.Diagnostics;
using System.Reflection;

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {
            AppDomain.CurrentDomain.AssemblyResolve += OnAssemblyResolve;

            ConsoleShell.Start("Hello", "", new string[0] { });
        }

        private static Assembly OnAssemblyResolve(object sender, ResolveEventArgs args)
        {
            Debugger.Launch();
            return null;
        }
    }
}
```
```
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.PowerShell.SDK" Version="7.0.0-rc.1" />
  </ItemGroup>

  <ItemGroup>
    <Reference Include="Microsoft.PowerShell.ConsoleHost">
      <HintPath>$(NuGetPackageRoot)\microsoft.powershell.consolehost\7.0.0-rc.1\runtimes\win\lib\netcoreapp3.1\Microsoft.PowerShell.ConsoleHost.dll</HintPath>
    </Reference>
  </ItemGroup>

</Project>
```
## Problem:
When run two import (`Import-Module C:\version1\assembly.dll; Import-Module C:\version2\assembly.dll`) with different versions of the assembly, I hit error of `Import-Module: Assembly with same name is already loaded` but the `AssemblyResolve` event was not raised.  

## Attempt 2: binding redirect through config file
```
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  <dependentAssembly>
    <assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
    <bindingRedirect oldVersion="0.0.0.0-9.3.3.0" newVersion="9.3.3.0" />
  </dependentAssembly>
</assemblyBinding>
```
## Problem:
Run into the same error above

## Attempt 3: run any external commands with `Invoke-Command -Session`, this basically out-sourced the command on a different domain.
## Problem:
This seems to be the only workaround I can come up with, but it's really inconvenient to jump back and forth PS sessions.

## Question:
Any better workaround?
Workarounds in loading assembly with different versions
issue
created
iSazonov
username_1
/cc @username_4 for information.
comment
created
SeeminglyScience
username_2
PowerShellEditorServices had some similar issues, @username_3 recently worked around that with `AssemblyLoadContext`'s.

[Here's the load context class](https://github.com/PowerShell/PowerShellEditorServices/blob/81ba0c0b8efb7ee879f2ede511390ba73b0bc86e/src/PowerShellEditorServices.Hosting/Internal/PsesLoadContext.cs).

And [here's where he sets it up for use](https://github.com/PowerShell/PowerShellEditorServices/blob/81ba0c0b8efb7ee879f2ede511390ba73b0bc86e/src/PowerShellEditorServices.Hosting/EditorServicesLoader.cs#L79-L109).
comment
created
rjmholt
username_3
Yeah, if you control the assembly load of your dependency, you can load it into a new assembly load context. In PSES we accomplish this as follows:

- Compile the top level module against only the APIs we want to share with the default ALC (this means the dependencies won't be exposed to PowerShell, only the API of your own module)
- Deploy the module with the dependency DLLs in a separate directory so that PowerShell won't find and load them itself
- Define a new custom ALC in a new class with logic to look for requested assemblies in this isolated dependency directory
- Add to the default ALC a new assembly resolve event handler for the top level DLL that we want to load into the default ALC (but with its dependencies in the custom ALC). This will load the top level assembly through the custom ALC into the default ALC (so it's loaded in both), but in such a way that its dependencies are only loaded into the custom ALC.

This assembly resolve setup means you don't need to use reflection anywhere, you just redirect loading through a new ALC at runtime. But it requires some layers:

- PowerShell (owns the default ALC)
- Assembly loaded immediately as part of module into the default ALC, where you set up custom ALC and resolve events. This is effectively a facade that does just enough to set up the ALCs and load your real API
- The top level API DLL to be loaded through the custom ALC into the default ALC
- Your dependencies, to be loaded into the custom ALC only

We've discussed getting this into PowerShell itself for modules, but it's quite complex and not planned currently, so unlikely to happen within the next 12 months at least.
comment
created
rjmholt
username_3
I should say that if you're rehosting PowerShell, you can condense the layers I describe above; you'll just have:

- Hosted PowerShell with the custom ALC
- The wrapping API to load into the default ALC (it can't clash with the Az modules' API)
- The clashing dependency to load into the custom ALC
comment
created
ritchxu
username_0
1. Why do we need the top level assembly **loaded in both default ALC and the custom ALC**?
2. Once our own module is loaded, if we intend to import Az modules and use it, it's going to be loaded to the default ALC. If so, how to avoid the assembly resolve event handler from kicking in? Do we simply `return null` to let the default ALC to resolve?
3. Dummy question: the following doesn't seem to work for dynamically-loaded dependencies, why would I need a subclass of `AssemblyLoadContext`?
```
$customContext = [System.Runtime.Loader.AssemblyLoadContext]::new("Test")
$onAssemblyResolve = {
    param($sender, $asmName)
    Write-Host "Resolving '$($asmName.Name)'"
    return $customContext.LoadFromAssemblyPath("$PSScriptRoot\bin\$($asmName.Name).dll")
}
[System.Runtime.Loader.AssemblyLoadContext]::Default.add_Resolving($onAssemblyResolve)
```
comment
created
daxian-dbw
username_4
I want to point out that it won't work well to load PowerShell into a custom ALC. PowerShell depends on `Assembly.LoadFrom` heavily especially in module loading, but `Assembly.LoadFrom` will load an assembly to the default ALC, so it will result in conflicts.
comment
created
SeeminglyScience
username_2
That's a scoping issue.  By the time your resolving handler is invoked, `$customContext` is no longer a variable.  You'd have to do something like `[System.Runtime.Loader.AssemblyLoadContext]::Default.add_Resolving($onAssemblyResolve.GetNewClosure())` but really you're better off doing it from C#.
comment
created
rjmholt
username_3
PowerShell has its own assembly resolve event that fires before yours. When you load the Az module, that loads its immediate assembly. That assembly then needs its dependencies, which aren't found in $PSHOME (.NET's app root here), so PowerShell's resolve event handler gets called, which looks in the directory where that first assembly is loaded.

So for example:

- `OurModule\`
  - `OurModule.psd1`
  - `OurModule.dll`
  - `isolatedDeps\`
      - `FacadeApi.dll`
      - `SharedDep.dll`
- `AzModule\`
    - `AzModule.psd1`
    - `AzModule.dll`
    - `SharedDep.dll`

When you load `OurModule`, you load `OurModule.dll` which creates the custom ALC and adds a resolve event that checks if `FacadeApi` is the requested assembly and if so looks in `isolatedDeps` to load `FacadeApi.dll` in the custom ALC and take the result of that and return it in the resolve event handler for the default ALC. The custom ALC just looks for anything its asked for in `isolatedDeps` and that's how it finds the `SharedDep.dll` in the `isolatedDeps` directory. The `OurModule` is loaded, PowerShell fires its resolve handler, looks in `OurModule\` and finds nothing, so we go to your resolve event, which only looks for the `FacadeApi.dll`.

When you load `AzModule`, whether before or after, PowerShell loads the module which directly loads `AzModule.dll`. That assembly depends on another particular version of `SharedDep.dll` (or could be the same one, doesn't matter). That's not loaded into the default ALC at any time, so always fires the resolve event, which PowerShell handles, looks for `SharedDep.dll` in `AzModule\`, finds it, and successfully loads.

So the two never clash. But it depends on the `FacadeApi.dll` existing and being unique to your module.

The other way to do it is to load your `SharedDep.dll` ahead of time in the custom ALC, but then you're forced to use reflection to call into its APIs.
comment
created
ritchxu
username_0
@username_3 I think I'm getting closer with your extremely useful explanation above.

Using the PSES as an example, if I understood correctly from the code, here is what happens:
1. We create the custom ALC
2. We load top-level module `Microsoft.PowerShell.EditorServices` using the custom ALC
3. We rely on the custom ALC to load dependencies (by the logic in custom ALC's overwritten `Load` method).

If I understood correctly, when I applies the same logic in my code, the overwritten `Load` was not invoked, **any clue how that could happen?**

Alternatively, I played with following approach instead:
```
testContext = new AssemblyLoadContext("Test");
binDirectory = @"C:\Test\bin";
AssemblyLoadContext.Default.Resolving += (AssemblyLoadContext defaultLoadContext, AssemblyName asmName) =>
{
     Console.WriteLine($"Resolving {asmName.Name}.dll");
     return testContext.LoadFromAssemblyPath(Path.Combine(binDirectory, $"{asmName.Name}.dll"));
};
```

This seems to be working, and here is what I assumed happening:
1. My own top-level module and its dependencies are loaded in `testContext`
2. When later importing `Az` module, PS will resolve all its dependencies to default ALC and no conflict occurred
comment
created
ritchxu
username_0
The problem with the approach above is, (at least what seems to be), if I import `Az` module first and say it loads a dependency `dep.1.dll` which my module has a different version of, say `dep.2.dll`. When `dep.2.dll` was later loaded, it seems it's loading `dep.1.dll` instead of going through my custom ALC's `Load` method, hence jeopardize the separation. Why wouldn't the custom ALC's `Load` method not invoked in this case?
comment
created
rjmholt
username_3
So the important details are:

- If you load an assembly into the default context (even if you loaded it via a custom ALC), it will now be the result of trying to load assemblies of the same name into the default ALC
- The key to our custom ALC logic is that we hook a resolve event to only respond to a specific name and version of an assembly, fire up a custom ALC around it and then load it into the default context. So that assembly we check for in the resolve event is the bridge between the two ALCs. What I've been calling the facade assembly/API. In your snippet, you're trying to load **all** assemblies firing a resolve event through the custom ALC into the default ALC, which defeats the purpose of the custom ALC. You need two layers; one to resolve the bridging assembly and one to load the conflicting dependencies in isolation, so they never get pulled into the default ALC.

For example:

```csharp
s_customLoadContext = new MyLoadContext(s_isolatedBinDir)
AssemblyLoadContext.Default.Resolving += (AssemblyLoadContext defaultLoadContext, AssemblyName asmName) =>
{
    Console.WriteLine($"Resolving {asmName.Name}.dll");
    if (asmName.Name != s_facadeAsmName
        || asmName.Version != s_facadeAsmVersion)
    {
        return null;
    }

    // Possibly s_customLoadContext.LoadFromAssemblyName(asmName) would work here too
    // and deduplicate knowledge of the bin dir path
    return s_customLoadContext.LoadFromAssemblyPath(Path.Combine(s_isolatedBinDir, $"{s_facadeAsmName}.dll"));
};

internal class MyLoadContext : AssemblyLoadContext
{
    private readonly string _dependencyDirPath;

    public MyLoadContext(string binDirPath)
    {
        _dependencyDirPath = binDirPath;
    }    

    string asmPath = Path.Join(_dependencyDirPath, $"{assemblyName.Name}.dll");

    if (File.Exists(asmPath))
    {
        return LoadFromAssemblyPath(asmPath);
    }

    return null;
}
```

Also, I'm not entirely sure why your custom ALC's load isn't being called, but the way I approached it was to log things everywhere to trace the logic
comment
created
rjmholt
username_3
One thought here: if an assembly is loaded into a context and you try to load another assembly of the same name into that context, it won't try to load the new one and there won't be any resolve event either. So it may be that you've already loaded the assembly you want into the default ALC, meaning no resolve event, meaning no load call on the custom ALC.
comment