Repository: QuantConnect/lean-cli
Pull request #541 (2025-01-30 04:19:19)

Fix mount points conflict causing empty project DLL in research environment
**Problem Description**  
When running `lean research`, the `/LeanCLI` directory inside the Docker container was missing all `.cs` source files, leaving only an generated `.csproj` by the tool. This caused `dotnet build` to emit the warning `CS2008: No source files specified`, resulting in an invalid project DLL (the content can be checked by `ilspycmd`) placed in ` /Lean/Launcher/bin/Debug/` that prevented referencing namespaced C# code of the project from Jupyter notebook.

**Root Cause**  
The Lean CLI attempted to mount the project directory to two paths in the container:  
1. `/LeanCLI` for C# compilation. This is needed for the correct compilation.
2. `/Lean/Launcher/bin/Debug/Notebooks` for Jupyter Lab working directory. This is also needed for accessing the host project directory content directly.

However, the `volumes` python dict uses host paths as keys. Subsequent mounts overwrote earlier ones, leaving only the last mount (to `/Lean/Launcher/bin/Debug/Notebooks`) active.

Additionally, the `_ensure_csproj_is_valid` method (called inside `get_basic_docker_config`) would generate temporary `.csproj` files and mounted them to `/LeanCLI`, which means we already have files mounted in `/LeanCLI` directory. Another relevant mount point that cause shadowing is the `config.json` for the `Notebooks/` directory, therefore we'd better have the mounting for `Notebooks/` prior to any other mounting in the `Notebooks/` directory to ensure the shadowing working as expected.

**Expected Behavior**

We need to have `/LeanCLI` to be compatible with backtest/live, and allow it to compile the project DLL correctly. **This compilation is still needed for research because in C# jupyter kernel, we can only load script files, those C# source files containing `namespace` are not allowed to be used directly in notebook cells and therefore can only be referenced by loading the corresponding project DLL.** User can then run `#r ../<project_name>.dll` after the `#load "../QuantConnect.csx"`.

Or even better, add a command to `commands` then we can `echo "Assembly.LoadFrom($project_name);" >> /Lean/Launcher/bin/Debug/QuantConnect.csx` to load the DLL automatically. But I am not sure if this would break anything, so for simplicity didn't do that in this PR.

**Chosen Solution** 
Replaced `volumes` with explicit `mounts` for the project directory and move the mounting logic prior to all other `Notebooks` related mounting.



Patch:
diff --git a/.gitignore b/.gitignore
index a90a066..3452cad 100644
--- a/.gitignore
+++ b/.gitignore
@@ -88,7 +88,7 @@ ipython_config.py
 # pyenv
 #   For a library or package, you might want to ignore these files since the code is
 #   intended to run in multiple environments; otherwise, check them in:
-# .python-version
+.python-version
 
 # pipenv
 #   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
diff --git a/lean/commands/research.py b/lean/commands/research.py
index 70061cf..9d96232 100644
--- a/lean/commands/research.py
+++ b/lean/commands/research.py
@@ -149,6 +149,14 @@ def research(project: Path,
                                                       research_image,
                                                       paths_to_mount)
 
+    # Mount project dir to the Notebooks directory first, avoid using volumes to prevent overwriting mounting logic for /LeanCLI
+    run_options["mounts"].append(Mount(
+        target=f"{LEAN_ROOT_PATH}/Notebooks",
+        source=str(project),
+        type="bind",
+        read_only=False
+    ))
+
     # Mount the config in the notebooks directory as well
     local_config_path = next(m["Source"] for m in run_options["mounts"] if m["Target"].endswith("config.json"))
     run_options["mounts"].append(Mount(target=f"{LEAN_ROOT_PATH}/Notebooks/config.json",
@@ -166,12 +174,6 @@ def research(project: Path,
     # Make Ctrl+C stop Jupyter Lab immediately
     run_options["stop_signal"] = "SIGKILL"
 
-    # Mount the project to the notebooks directory
-    run_options["volumes"][str(project)] = {
-        "bind": f"{LEAN_ROOT_PATH}/Notebooks",
-        "mode": "rw"
-    }
-
     # Allow notebooks to be embedded in iframes
     run_options["commands"].append("mkdir -p ~/.jupyter")
     run_options["commands"].append(


Test patch:
diff --git a/tests/commands/test_research.py b/tests/commands/test_research.py
index 9dd57e8..688483e 100644
--- a/tests/commands/test_research.py
+++ b/tests/commands/test_research.py
@@ -280,3 +280,43 @@ def test_research_runs_lean_container_with_paths_to_mount() -> None:
 
     assert mount is not None
     assert mount["Target"] == "/Files/file.json"
+
+def test_research_mounts_project_directory_to_leancli_and_notebooks() -> None:
+    create_fake_lean_cli_directory()
+
+    docker_manager = mock.MagicMock()
+    container.initialize(docker_manager)
+
+    project_dir = Path.cwd() / "CSharp Project"
+    (project_dir / "Main.cs").touch()
+    original_csproj = project_dir / "CSharp Project.csproj"
+    original_csproj.write_text("""
+<Project>
+    <PropertyGroup></PropertyGroup>
+    <ItemGroup>
+        <PackageReference Include="QuantConnect.Lean.Engine" Version="2.5.*" />
+    </ItemGroup>
+</Project>""")
+
+    result = CliRunner().invoke(lean, ["research", "CSharp Project"])
+
+    assert result.exit_code == 0
+
+    docker_manager.run_image.assert_called_once()
+    args, kwargs = docker_manager.run_image.call_args
+
+    leancli_volume = next(((k, v) for (k, v) in kwargs["volumes"].items() if v['bind'] == f"/LeanCLI"), None)
+    notebooks_mount = next((m for m in kwargs["mounts"] if m["Target"] == f"{LEAN_ROOT_PATH}/Notebooks"), None)
+
+    assert leancli_volume is not None, "/LeanCLI is not mounted"
+    assert notebooks_mount is not None, "/Notebooks is not mounted"
+    assert leancli_volume[0] == str(project_dir)
+    assert notebooks_mount["Source"] == str(project_dir)
+
+    temp_csproj_mounts = [
+        m for m in kwargs["mounts"]
+        if m["Target"].startswith(f"/LeanCLI") and m["Target"].endswith(".csproj")
+    ]
+
+    assert len(temp_csproj_mounts) > 0, "No temporary csproj file mounts detected"
+    assert all(m["Source"] != str(original_csproj) for m in temp_csproj_mounts), "Temporary csproj did not correctly overwrite user file"