Repository: pyodide/pyodide

Issue:
CPython stack frame size estimation
The JavaScript stack capacity is limited in web browsers. In turn, this imposes a limit on the maximum CPython stack size and maximum recursion depth in a python program.

Some python applications hit the stack limit. One example of such python module is `jedi` (an autocompletion, static analysis and refactoring library).

Although we cannot do anything about the JavaScript stack limit in web browsers, we noticed that the stack frame size of CPython can be optimized. By reducing the stack frame size, it is possible to achieve higher recursion depth.

We observe that the stack frame is approx 18% larger in pyodide version 0.17.0 (as of rev. c327405), compared to pyodide versions 0.14.1 and 0.15.0.
However, we see an improvement in this aspect when comparing version 0.17.0 to version 0.16.1.

We use this script to approximate the CPython stack frame size:
```javascript
loadPyodide()
measure_available_js_stack_depth = () => {
    let depth = 0;
    function recurse() { depth += 1; recurse(); }
    try { recurse(); } catch (err) { }
    return depth;
}; pyodide.runPython("import js\ndef recurse(n):\n    return js.measure_available_js_stack_depth() if n==0 else recurse(n-1)\n(recurse(0)-recurse(100))/100")
```

The code snippet above is to be pasted to the JavaScript console in the `build/test.html` page. The script outputs a single number, which is the relative stack frame size of CPython. The smaller the better.

The pyodide team [has shown an interest](https://github.com/pyodide/pyodide/issues/1331#issuecomment-823483383) to integrate stack frame size estimation into CI to hunt down regressions.

This issue was created to facilitate any discussion on that topic.


Discussion:
Great! Would you be willing to open a PR adding this script to the CI? I think the best way is to copy the existing test-core-chrome and test-core-firefox jobs and swap out the command.
That's very useful thanks!

> I think the best way is to copy the existing test-core-chrome and test-core-firefox jobs and swap out the command.

Maybe rather add it as part of benchmarks? We already have things like load time there, and the generated .json artifact would allow to follow the evolution with time.
Would someone be interested in opening a question in https://discuss.python.org/ (section users) and asking if there is anything else we can do to build CPython for an environment with a small stack size? 

I'm open to compiling with `-Os` (mentioned earlier) if the performance impact is not too large. 
> I'm open to compiling with `-Os` (mentioned earlier) if the performance impact is not too large.

This is our patch that enables size optimizations and reduces the relative CPython stack frame size from `47.33` to `41.34` (tested on revision 958724101a9cd6a1181feadcec811f26153ff828):

```diff
diff --git a/cpython/Makefile b/cpython/Makefile
index 3c0220b..411e9cc 100644
--- a/cpython/Makefile
+++ b/cpython/Makefile
@@ -111,6 +111,7 @@ $(BUILD)/Makefile: $(BUILD)/.patched $(ZLIBBUILD)/.configured $(SQLITEBUILD)/lib
                cd $(BUILD); \
                CONFIG_SITE=./config.site READELF=true emconfigure \
                  ./configure \
+                         OPT="-fwrapv -Os -Wall" \
                          CFLAGS="-fPIC -sEMULATE_FUNCTION_POINTER_CASTS=1" \
                          CPPFLAGS="-I$(SQLITEBUILD) -I$(BZIP2BUILD) -I$(ZLIBBUILD)" \
                          --without-pymalloc \
```
A welcomed side-effect of this change is reduced size of `pyodide.asm.wasm` from 10212kB to 9316kB.

It would be great if the size optimization were set by default for CPython. We haven't evaluated the performance impacts, though.

Could the optimization flag be passed as an argument to the top-level `Makefile`? This way, it would be easy to do custom pyodide builds fitted for a particular use-case.
> Could the optimization flag be passed as an argument to the top-level `Makefile`?

+1, a PR doing this would be very welcome.
> It would be great if the size optimization were set by default for CPython. We haven't evaluated the performance impacts, though.

Yeah, that's consistent with my earlier experiments in https://github.com/pyodide/pyodide/issues/1365#issuecomment-807364368

> Could the optimization flag be passed as an argument to the top-level Makefile?

I think it should have used https://github.com/pyodide/pyodide/blob/ffa99153cc4ccb233ad11d3cd58108f06707d01f/Makefile.envs#L28

and we could change that to `Os` (or `Oz`).

And yes ideally we should move all of such global config settings to `Makefile.envs` than optionally use the corresponding env variable if it's definied as in https://github.com/pyodide/pyodide/blob/ffa99153cc4ccb233ad11d3cd58108f06707d01f/Makefile.envs#L18-L19
A single stack frame for a simple Python function call looks like this:
```js
    at pyodide.asm.wasm:0x1f08e1
    at _PyFunction_Vectorcall (pyodide.asm.wasm:0x1f06b2)
    at byn$fpcast-emu$_PyFunction_Vectorcall (pyodide.asm.wasm:0x7eec18)
    at pyodide.asm.wasm:0x2d9f0d
    at _PyEval_EvalFrameDefault (pyodide.asm.wasm:0x2d7383)
    at byn$fpcast-emu$_PyEval_EvalFrameDefault
```
So the first thing of note is that the `byn$fpcast-emu` calls are HUGE stack hogs: they take 63 64 bit integers as their argument. I wonder if we could change the fpcast system to pass instead a giant struct. That struct would then be allocated in the "argument stack" and wouldn't contribute to call stack overflows -- I think we have a lot more argument stack space if we need it, and we can control how much argument stack space we allocate for ourselves.

Another option is we could try to disable fpcast emulation for these particular functions.

Another thing we could do is make `_PyEval_EvalFrame` directly call `_PyEval_EvalFrameDefault`, short circuiting the `tstate->interp->eval_frame` mechanism. Not likely anyone can run a JIT in Pyodide anyways. That would drop one `byn$fpcat-emu` frame and one smaller stack frame.
This stack usage measurement doesn't seem to work in firefox.

Resolving patch:
diff --git a/benchmark/stack_usage.py b/benchmark/stack_usage.py
new file mode 100644
--- /dev/null
+++ b/benchmark/stack_usage.py
@@ -0,0 +1,78 @@
+import pytest
+
+
+@pytest.fixture(scope="session")
+def print_info():
+    headings = [
+        "browser",
+        "py_usage",
+        "js_depth",
+        "py_depth",
+        "js_depth/py_usage",
+        "js_depth/py_depth",
+    ]
+    fmt = "## {{:{:d}s}}  {{:{:d}.2f}}  {{:{:d}g}}  {{:{:d}g}}  {{:{:d}g}}  {{:{:d}g}}".format(
+        *map(len, headings)
+    )
+    printed_heading = False
+
+    def print_info(*args):
+        nonlocal printed_heading
+        if not printed_heading:
+            printed_heading = True
+            print("## " + "  ".join(headings))
+        print(fmt.format(*args))
+
+    yield print_info
+
+
+@pytest.mark.skip_refcount_check
+@pytest.mark.skip_pyproxy_check
+def test_stack_usage(selenium, print_info):
+    res = selenium.run_js(
+        """
+        window.measure_available_js_stack_depth = () => {
+            let depth = 0;
+            function recurse() { depth += 1; recurse(); }
+            try { recurse(); } catch (err) { }
+            return depth;
+        };
+        let py_usage = pyodide.runPython(`
+            from js import measure_available_js_stack_depth
+            def recurse(n):
+                return measure_available_js_stack_depth() if n==0 else recurse(n-1)
+            (recurse(0)-recurse(100))/100
+        `);
+        let js_depth = measure_available_js_stack_depth();
+        window.py_depth = [0];
+        try {
+        pyodide.runPython(`
+        import sys
+        from js import py_depth
+        sys.setrecursionlimit(2000)
+        def infiniterecurse():
+            py_depth[0] += 1
+            infiniterecurse()
+        infiniterecurse()
+        `);
+        } catch {}
+
+        py_depth = py_depth[0];
+        return [
+            py_usage,
+            js_depth,
+            py_depth,
+            Math.floor(js_depth/py_usage),
+            Math.floor(js_depth/py_depth),
+        ]
+        """
+    )
+    # "py_usage",
+    # "js_depth",
+    # "py_depth",
+    # "js_depth/py_usage",
+    # "js_depth/py_depth",
+
+    print_info(selenium.browser, *res)
+
+    selenium.clean_logs()


Test patch: