Repository: QuantConnect/lean-cli (python)

Issue:
pydev debugger: CRITICAL WARNING
Create Python project, and debug it via

```
lean backtest ".\My Project\" --debug ptvsd
```

We get the following message in the console:

```
20240429 17:53:54.923 TRACE:: DebuggerHelper.Initialize(): waiting for PTVSD debugger to attach at localhost:5678...
-------------------------------------------------------------------------------
pydev debugger: CRITICAL WARNING: This version of python seems to be incorrectly compiled (internal generated filenames
are not absolute)
pydev debugger: The debugger may still function, but it will work slower and may miss breakpoints.
pydev debugger: Related bug: http://bugs.python.org/issue1666807
-------------------------------------------------------------------------------
pydev debugger: Unable to find real location for: /root/.local/lib/python3.11/site-packages
pydev debugger: Unable to find real location for: <frozen posixpath>
pydev debugger: Unable to find real location for: <frozen genericpath>
pydev debugger: Unable to find real location for: <frozen abc>
pydev debugger: Unable to find real location for: <string>
20240429 17:54:08.036 TRACE:: DebuggerHelper.Initialize(): started
20240429 17:54:08.037 TRACE:: Config.GetValue(): algorithm-creation-timeout - Using default value: 90
20240429 17:54:08.047 TRACE:: AlgorithmPythonWrapper(): Python version 3.11.7 (main, Dec 15 2023, 18:24:52) [GCC
11.2.0]: Importing python module main
pydev debugger: Unable to find real location for: <frozen importlib._bootstrap>
pydev debugger: Unable to find real location for: <frozen importlib._bootstrap_external>
pydev debugger: Unable to find real location for: <frozen zipimport>
pydev debugger: Unable to find real location for: <frozen _collections_abc>
pydev debugger: Unable to find real location for: <frozen os>
pydev debugger: Unable to find real location for: <frozen io>
pydev debugger: Unable to find real location for: <frozen codecs>
```

We can attach the process and debug the 
![Captura de ecrã 2024-04-29 185442](https://github.com/QuantConnect/lean-cli/assets/6889033/fc7e935b-30b4-401a-8673-398e718258cd)
algorithm:



Pull request:
Enable local Python debugging with DebugPy
- Enable local Python debugging with DebugPy.
- Warn PTVSD deprecation:
  - PTVSD has been[ deprecated](https://github.com/microsoft/ptvsd) for 4+ years and DebugPy is its replacement.
  - PTVSD does not support Python 3.11, which is the cause of the issue.
  - We don't remote PTVSD yet but warn of it's deprecation and recommend using DebugPy instead.

Tested by debugging using `--debug debugpy` and following the nstructions in "[Python and VS Code](https://www.quantconnect.com/docs/v2/lean-cli/backtesting/debugging#03-Python-and-VS-Code)".
The instructions should be modified to reflect that using DebugPy is recommented now.

Closes #451 

Patch:
diff --git a/README.md b/README.md
index df71f3e..88459e2 100644
--- a/README.md
+++ b/README.md
@@ -143,7 +143,7 @@ Usage: lean backtest [OPTIONS] PROJECT
 Options:
   --output DIRECTORY              Directory to store results in (defaults to PROJECT/backtests/TIMESTAMP)
   -d, --detach                    Run the backtest in a detached Docker container and return immediately
-  --debug [pycharm|ptvsd|vsdbg|rider|local-platform]
+  --debug [pycharm|ptvsd|debugpy|vsdbg|rider|local-platform]
                                   Enable a certain debugging method (see --help for more information)
   --data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit]
                                   Update the Lean configuration file to retrieve data from the given historical provider
diff --git a/lean/components/docker/lean_runner.py b/lean/components/docker/lean_runner.py
index 4bbe384..7c08ba1 100644
--- a/lean/components/docker/lean_runner.py
+++ b/lean/components/docker/lean_runner.py
@@ -107,8 +107,8 @@ class LeanRunner:
         # Add known additional run options from the extra docker config
         self.parse_extra_docker_config(run_options, extra_docker_config)
 
-        # Set up PTVSD debugging
-        if debugging_method == DebuggingMethod.PTVSD:
+        # Set up PTVSD or DebugPy debugging
+        if debugging_method == DebuggingMethod.PTVSD or debugging_method == DebuggingMethod.DebugPy:
             run_options["ports"]["5678"] = "5678"
 
         # Set up VSDBG debugging
diff --git a/lean/models/utils.py b/lean/models/utils.py
index fd53afc..7c868d0 100644
--- a/lean/models/utils.py
+++ b/lean/models/utils.py
@@ -22,7 +22,8 @@ class DebuggingMethod(Enum):
     PTVSD = 2
     VSDBG = 3
     Rider = 4
-    LocalPlatform  = 5
+    LocalPlatform = 5
+    DebugPy = 6
 
     def get_internal_name(self) -> str:
         """Returns the LEAN debugging method that should be used for the current enum member.
@@ -32,6 +33,7 @@ class DebuggingMethod(Enum):
         return {
             DebuggingMethod.PyCharm: "PyCharm",
             DebuggingMethod.PTVSD: "PTVSD",
+            DebuggingMethod.DebugPy: "DebugPy",
             DebuggingMethod.LocalPlatform: "DebugPy" # QC -> DebugPy, If its Python it uses DebugPy, if its C# LEAN safely ignores DebugPy
         }.get(self, "LocalCmdline")
 


Test patch:
diff --git a/lean/commands/backtest.py b/lean/commands/backtest.py
index 832fe22..9c0836d 100644
--- a/lean/commands/backtest.py
+++ b/lean/commands/backtest.py
@@ -234,7 +234,7 @@ def _migrate_csharp_csproj(project_dir: Path) -> None:
               default=False,
               help="Run the backtest in a detached Docker container and return immediately")
 @option("--debug",
-              type=Choice(["pycharm", "ptvsd", "vsdbg", "rider", "local-platform"], case_sensitive=False),
+              type=Choice(["pycharm", "ptvsd", "debugpy", "vsdbg", "rider", "local-platform"], case_sensitive=False),
               help="Enable a certain debugging method (see --help for more information)")
 @option("--data-provider-historical",
               type=Choice([dp.get_name() for dp in cli_data_downloaders], case_sensitive=False),
@@ -331,6 +331,11 @@ def backtest(project: Path,
     elif debug == "ptvsd":
         debugging_method = DebuggingMethod.PTVSD
         _migrate_python_vscode(algorithm_file.parent)
+        logger.warn("The PTVSD debugging method is deprecated and might be removed in a future version of LEAN. "
+                    "Consider using DebugPy instead.")
+    elif debug == "debugpy":
+        debugging_method = DebuggingMethod.DebugPy
+        _migrate_python_vscode(algorithm_file.parent)
     elif debug == "vsdbg":
         debugging_method = DebuggingMethod.VSDBG
         _migrate_csharp_vscode(algorithm_file.parent)
diff --git a/tests/commands/test_backtest.py b/tests/commands/test_backtest.py
index 9a3a373..80d4826 100644
--- a/tests/commands/test_backtest.py
+++ b/tests/commands/test_backtest.py
@@ -283,6 +283,8 @@ def _ensure_rider_debugger_config_files_exist(project_dir: Path) -> None:
                                                     ("PTVSD", DebuggingMethod.PTVSD),
                                                     ("vsdbg", DebuggingMethod.VSDBG),
                                                     ("VSDBG", DebuggingMethod.VSDBG),
+                                                    ("debugpy", DebuggingMethod.DebugPy),
+                                                    ("DebugPy", DebuggingMethod.DebugPy),
                                                     ("rider", DebuggingMethod.Rider),
                                                     ("Rider", DebuggingMethod.Rider)])
 def test_backtest_passes_correct_debugging_method_to_lean_runner(value: str, debugging_method: DebuggingMethod) -> None:
@@ -348,7 +350,8 @@ def test_backtest_auto_updates_outdated_python_pycharm_debug_config() -> None:
     assert workspace_xml.find(".//mapping[@remote-root='/Lean/Launcher/bin/Debug']") is None
 
 
-def test_backtest_auto_updates_outdated_python_vscode_debug_config() -> None:
+@pytest.mark.parametrize("value", ["ptvsd", "debugpy"])
+def test_backtest_auto_updates_outdated_python_vscode_debug_config(value) -> None:
     create_fake_lean_cli_directory()
 
     lean_config_manager = container.lean_config_manager
@@ -380,7 +383,7 @@ def test_backtest_auto_updates_outdated_python_vscode_debug_config() -> None:
         ]
     }))
 
-    result = CliRunner().invoke(lean, ["backtest", "Python Project", "--debug", "ptvsd"])
+    result = CliRunner().invoke(lean, ["backtest", "Python Project", "--debug", value])
 
     assert result.exit_code == 0