Repository: vscode
Issue: VSCode is taking too long to execute first python test
Labels: bug,perf,testing,python


Type: <b>Bug</b>

VSCode takes too long to execute first python test after refreshing the tests. This time increases quadratically with the number of tests in working directory. A side effect of this is that in repos with large number of tests especially parametrized tests, it drives the CPU consumption of extension host process to 100% and causes VSCode window to freeze. It only works again after reloading the window.

|Number of tests|Time taken to run first test (s)
|---|---|
|10000|8.6|
|20000|14.1|
|30000|46.1|
|40000|94.6|
|50000|194.5|
|60000|344.3|

![Image](https://github.com/user-attachments/assets/361c850f-51f7-43eb-ad64-82eab0996883)

Note that the time taken by the run_adapter script to discover tests is proportional to the number of tests so this issue is in the VSCode UI itself.

<b>Reproducer</b>

Workspace settings:
```
{
    "python.testing.pytestEnabled": true,
    "python.testing.cwd": "src/python",
}
```
Directory structure:
```
.
└── src
    └── python
        ├── __pycache__
        │   ├── test_example.cpython-311-pytest-7.4.3.pyc
        │   └── test_sample.cpython-311-pytest-7.4.3.pyc
        ├── test_example.py
        └── test_sample.py

```
`test_example.py` contains:
```
import pytest

count = 10000

@pytest.mark.parametrize("x", [i for i in range(count)])
def test_parametrize(x):
    pass

```
`test_sample.py` contains:
```
def test_a():
    pass

```
Steps to reproduce:
1. Click on refresh tests icon.
2. Once the test tree loads and `test_a` is present, click on the run test icon.


VS Code version: Code 1.93.0 (4849ca9bdf9666755eb463db297b69e5385090e3, 2024-09-04T13:02:38.431Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 4.18.0-553.16.1.el8_10.x86_64
Remote OS version: Linux x64 4.18.0-553.16.1.el8_10.x86_64
Remote OS version: Linux x64 4.18.0-553.16.1.el8_10.x86_64
Remote OS version: Linux x64 4.18.0-553.16.1.el8_10.x86_64
Remote OS version: Linux x64 4.18.0-553.16.1.el8_10.x86_64

<details>
<summary>System Info</summary>

|Item|Value|
|---|---|
|CPUs|12th Gen Intel(R) Core(TM) i9-12900 (24 x 2419)|
|GPU Status|2d_canvas: enabled<br>canvas_oop_rasterization: enabled_on<br>direct_rendering_display_compositor: disabled_off_ok<br>gpu_compositing: enabled<br>multiple_raster_threads: enabled_on<br>opengl: enabled_on<br>rasterization: enabled<br>raw_draw: disabled_off_ok<br>skia_graphite: disabled_off<br>video_decode: enabled<br>video_encode: enabled<br>vulkan: disabled_off<br>webgl: enabled<br>webgl2: enabled<br>webgpu: enabled<br>webnn: disabled_off|
|Load (avg)|undefined|
|Memory (System)|31.70GB (17.77GB free)|
|Process Argv||
|Screen Reader|no|
|VM|0%|

|Item|Value|
|---|---|
|Remote|SSH: dev-hyd|
|OS|Linux x64 4.18.0-553.16.1.el8_10.x86_64|
|CPUs|Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz (32 x 3388)|
|Memory (System)|376.07GB (135.51GB free)|
|VM|0%|
</details><details><summary>Extensions (7)</summary>

Extension|Author (truncated)|Version
---|---|---
remote-ssh|ms-|0.115.0
remote-ssh-edit|ms-|0.87.0
remote-explorer|ms-|0.4.3
black-formatter|ms-|2024.4.0
debugpy|ms-|2024.12.0
python|ms-|2024.14.1
vscode-pylance|ms-|2024.10.1


</details>
<!-- generated by issue reporter -->