Repository: pyodide/pyodide

Issue:
Update scipy version
Currently we are using scipy 0.17 from 2016 because [later versions could include f90](https://github.com/scipy/scipy/issues/2829#issuecomment-223764054) and we don't know how to compile it.

However, I quickly looked at scipy git log, and the mean edit frequency of for fortran files in scipy is 6 years, so even if f90 is now accepted now, there was no active modernization effort from f77. Which means that later scipy versions could potentially be built, and we could update it. Latest version would be ideal so that we could contribute fixes upstream if needed.

Though the problem is that until dynamic linking is not resolved https://github.com/iodide-project/pyodide/issues/240, if newer versions add additional modules that use LAPACK it will increase the binary size which is not desirable.


Discussion:
The current status on this is that it's possible to update scipy, but later versions use BLAS/LAPACK in a larger number of modules, which would translate to even bigger package sizes until we address it via https://github.com/iodide-project/pyodide/issues/240 or https://github.com/iodide-project/pyodide/issues/874
Any updates on when scipy will be updated? Looking to run a function (https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.cossin.html) which is available from scipy 1.5.0
Same thing. I'm lacking functions from new scipy =) 
To move this forward, we are working on compiling SciPy with LFortran: https://gitlab.com/lfortran/lfortran/-/issues/582. We can parse most of the files, some require either minor modifications or (preferably) a few fixes in our fixed-form parser.

I figured the best way would be to get some simplest self-contained part of SciPy working first. It seems the simplest part might be `stats/statlib`: https://gitlab.com/lfortran/lfortran/-/issues/581. It looks like we can parse those files and we are now working on the semantics. It looks like we are not far off, but we still have to implement a few things to get all semantics working. Once that works, we'll see what needs to be fixed in our LLVM backend, probably a few things also.

Once we can generate a correct LLVM IR code for those subroutines, what would be the next step? I assume we select the Wasm backend in the LLVM to generate Wasm. LFortran's runtime library is mostly written in Fortran, so that should translate nicely. There are some things that reach into C (such as file IO), but I think those are not needed for SciPy.

What about things like math functions such as `sin(x)`? We have two versions: one in pure Fortran, which is performance first, the accuracy is roughly 1e-15 relative accuracy and the argument range is roughly `|x| < 1e10`. If `x` is outside that range, it will return incorrect results. These functions are meant for Release mode, when speed is needed. Then we have accuracy first version which are much slower, but those simply call into `libc`, so depend on C. I assume we can add special handling for Wasm into our LLVM backend to call some accurate `sin(x)` implementation that is available for Wasm from somewhere. What is the best approach here?

This is obviously a longer term project, but we will eventually get there. Important is to start now, get simple things working all the way, and then eventually be able to compile all of SciPy. If there is anybody who wants to help us with this work, please let us know. We are always looking for new contributors. I created a general issue to add a Wasm backend to LFortran here: https://gitlab.com/lfortran/lfortran/-/issues/583.
Great to hear that @certik ! Having alternative compiler for scipy would be very useful, already on x86_64.

> Once we can generate a correct LLVM IR code for those subroutines, what would be the next step? I assume we select the Wasm backend in the LLVM to generate Wasm.

I assume so but I'm not familiar with LLVM. Looks like @StarGate01 managed to make that part work in https://github.com/pyodide/pyodide/issues/184#issuecomment-616948624

> What about things like math functions such as sin(x)? .. What is the best approach here?

Well the scipy test suite  would ideally need to pass on x86_64 (or have its tolerances adjusted). So that will tell how critical it would be. In Pyodide however we currently still have a number of failures, in particular due to an issue with float underflow/overflow handling https://github.com/pyodide/pyodide/issues/156

> Important is to start now, get simple things working all the way, and then eventually be able to compile all of SciPy.

Absolutely, thanks for starting this discussion!

To give a bit more context. Even once there is a compiler that is able to compile Fortran for WASM (related https://github.com/pyodide/pyodide/issues/184) Pyodide has other challenges for building scipy. Due to  function signature mismatches in dynamic linking that apparently work in non WASM environment but that make WASM crash arbitrarily, we currently [have lots of patches](https://github.com/pyodide/pyodide/tree/main/packages/scipy/patches) to try to work around that thanks to work done by @joemarshall . When that makes sense those would need to be contributed to scipy in the long term. At least that's an issue with scipy + CBLAS + CLAPACK + libf2c. It might not be a problem with scipy + BLAS + LAPACK (in Fortran), not sure. Also ideally we would need to switch to a more optimized BLAS #227  in the future.

Also congrats for the progress made on the LFortran project!
Also to give a general status update on this issue the latest WIP attempt to update is in https://github.com/pyodide/pyodide/pull/1293 but it requires lots of time and effort to fix all the signature mismatches and port patches. 
@rth, awesome, thanks a lot for the tips. It looks like a good first step for us is to:

* Compile the `stats/statlib` with just the regular CPU LLVM backends (x86, arm) and make sure scipy tests pass.

Although perhaps minor for your efforts, that would be a huge milestone for us. I think that's a necessary milestone for any further work and we are not that far off. After that works, we can then work on two things in parallel:

* Implement more Fortran features to make it work for other SciPy modules also
* Get `stats/statlib` working in the browser by reusing the f2py SciPy interface to it and creating a simple Python project using it. 
 
The last point would ensure all the pieces are working with regards to WebAssembly and Pyodide, and people can start using LFortran for simpler Fortran/Python projects. Then to get SciPy working, "all" we need to do is to implement the rest of the Fortran features that other SciPy modules use.

We are looking for more contributors to help us get over the finish line sooner. For example a huge help would be to iron out the WebAssembly and f2py specific interface issues with regards to LFortran, any potential user can help us with that. We can start this work today (update: we literally started today: https://gitlab.com/lfortran/lfortran/-/merge_requests/1385) on just some simple Fortran code and get it working. If anybody reading this is interested in using Fortran in the browser, please get in touch, I am happy to get you started.

Resolving patch:
diff --git a/pyodide-build/pyodide_build/_f2c_fixes.py b/pyodide-build/pyodide_build/_f2c_fixes.py
--- a/pyodide-build/pyodide_build/_f2c_fixes.py
+++ b/pyodide-build/pyodide_build/_f2c_fixes.py
@@ -1,173 +1,259 @@
 import re
+import subprocess
+from textwrap import dedent  # for doctests
+from typing import List, Iterable, Iterator, Tuple
+from pathlib import Path
 
 
-def fix_f2c_clapack_calls(f2c_output_name: str):
-    """Fix F2C CLAPACK calls
+def fix_f2c_output(f2c_output_path: str):
+    """
+    This function is called on the name of each C output file. It fixes up the C
+    output in various ways to compensate for the lack of f2c support for Fortan
+    90 and Fortran 95.
+    """
+    f2c_output = Path(f2c_output_path)
+    if f2c_output.name == "lapack_extras.c":
+        # dfft.c has a bunch of implicit cast args coming from functions copied
+        # out of future lapack versions. fix_inconsistent_decls will fix all
+        # except string to int.
+        subprocess.check_call(
+            [
+                "patch",
+                str(f2c_output_path),
+                f"../../patches/fix-implicit-cast-args-from-newer-lapack.patch",
+            ]
+        )
+
+    with open(f2c_output, "r") as f:
+        lines = f.readlines()
+    if "id_dist" in f2c_output_path:
+        # Fix implicit casts in id_dist.
+        lines = fix_inconsistent_decls(lines)
+    if "odepack" in f2c_output_path or f2c_output.name == "mvndst.c":
+        # Mark all but one declaration of each struct as extern.
+        if f2c_output.name == "blkdta000.c":
+            # extern marking in blkdata000.c doesn't work properly so we let it
+            # define the one copy of the structs. It doesn't talk about lsa001
+            # at all though, so we need to add a definition of it.
+            lines.append(
+                """
+                struct {    doublereal rownd2, pdest, pdlast, ratio, cm1[12], cm2[5], pdnorm;
+                    integer iownd2[3], icount, irflag, jtyp, mused, mxordn, mxords;
+                } lsa001_;
+                """
+            )
+        else:
+            add_externs_to_structs(lines)
+
+    if f2c_output.name in [
+        "wrap_dummy_g77_abi.c",
+        "_lapack_subroutine_wrappers.c",
+        "_blas_subroutine_wrappers.c",
+        "_flapack-f2pywrappers.c",
+    ]:
+        lines = remove_ftnlen_args(lines)
+
+    with open(f2c_output, "w") as f:
+        f.writelines(lines)
+
+
+def prepare_doctest(x):
+    return dedent(x).strip().split("\n")
+
+
+def remove_ftnlen_args(lines: List[str]) -> List[str]:
+    """
+    Functions with "character" arguments have these extra ftnlen arguments at
+    the end (which are never used). Other places declare these arguments as
+    "integer" which don't get length arguments. This automates the removal of
+    the problematic arguments.
+
+    >>> print("\\n".join(remove_ftnlen_args(prepare_doctest('''
+    ...     /* Subroutine */ int chla_transtypewrp__(char *ret, integer *trans, ftnlen
+    ...     	ret_len)
+    ... '''))))
+    /* Subroutine */ int chla_transtypewrp__(char *ret, integer *trans)
+
+    >>> print("\\n".join(remove_ftnlen_args(prepare_doctest('''
+    ...     /* Subroutine */ int clanhfwrp_(real *ret, char *norm, char *transr, char *
+    ...     	uplo, integer *n, complex *a, real *work, ftnlen norm_len, ftnlen
+    ...     	transr_len, ftnlen uplo_len)
+    ... '''))))
+    /* Subroutine */ int clanhfwrp_(real *ret, char *norm, char *transr, char * uplo, integer *n, complex *a, real *work)
+    """
+    new_lines = []
+    for line in regroup_lines(lines):
+        if line.startswith("/* Subroutine */"):
+            line = re.sub(r",\s*ftnlen [a-z]*_len", "", line)
+        new_lines.append(line)
+    return new_lines
+
+
+def add_externs_to_structs(lines: List[str]):
+    """
+    The fortran "common" keyword is supposed to share variables between a bunch
+    of files. f2c doesn't handle this correctly (it isn't possible for it to
+    handle it correctly because it only looks one file at a time).
+
+    We mark all the structs as externs and then (separately) add one non extern
+    version to each file.
+    >>> lines = prepare_doctest('''
+    ...     struct {    doublereal rls[218];
+    ...         integer ils[39];
+    ...     } ls0001_;
+    ...     struct {    doublereal rlsa[22];
+    ...         integer ilsa[9];
+    ...     } lsa001_;
+    ...     struct {    integer ieh[2];
+    ...     } eh0001_;
+    ... ''')
+    >>> add_externs_to_structs(lines)
+    >>> print("\\n".join(lines))
+    extern struct {    doublereal rls[218];
+        integer ils[39];
+    } ls0001_;
+    extern struct {    doublereal rlsa[22];
+        integer ilsa[9];
+    } lsa001_;
+    extern struct {    integer ieh[2];
+    } eh0001_;
+    """
+    for idx, line in enumerate(lines):
+        if line.startswith("struct"):
+            lines[idx] = "extern " + lines[idx]
 
-    f2c compiles code with fortran linkage, which means that
-    strings are passed as char* plus extra length argument at
-    the end.
 
-    CLAPACK uses C null terminated strings.
+def regroup_lines(lines: Iterable[str]) -> Iterator[str]:
+    """
+    Make sure that functions and declarations have their argument list only on
+    one line.
+
+    >>> print("\\n".join(regroup_lines(prepare_doctest('''
+    ...     /* Subroutine */ int clanhfwrp_(real *ret, char *norm, char *transr, char *
+    ...     	uplo, integer *n, complex *a, real *work, ftnlen norm_len, ftnlen
+    ...     	transr_len, ftnlen uplo_len)
+    ...     {
+    ...        static doublereal psum[52];
+    ...        extern /* Subroutine */ int dqelg_(integer *, doublereal *, doublereal *,
+    ...            doublereal *, doublereal *, integer *);
+    ... '''))))
+    /* Subroutine */ int clanhfwrp_(real *ret, char *norm, char *transr, char * uplo, integer *n, complex *a, real *work, ftnlen norm_len, ftnlen transr_len, ftnlen uplo_len)
+    {
+       static doublereal psum[52];
+       extern /* Subroutine */ int dqelg_(integer *, doublereal *, doublereal *, doublereal *, doublereal *, integer *);
+
+    """
+    line_iter = iter(lines)
+    for line in line_iter:
+        if not "/* Subroutine */" in line:
+            yield line
+            continue
+
+        is_definition = line.startswith("/* Subroutine */")
+        stop = ")" if is_definition else ";"
+        if stop in line:
+            yield line
+            continue
+
+        sub_lines = [line.rstrip()]
+        for line in line_iter:
+            sub_lines.append(line.strip())
+            if stop in line:
+                break
+        joined_line = " ".join(sub_lines)
+        if is_definition:
+            yield joined_line
+        else:
+            yield from (x + ";" for x in joined_line.split(";")[:-1])
+
+
+def fix_inconsistent_decls(lines: List[str]) -> List[str]:
+    """
+    Fortran functions in id_dist use implicit casting of function args which f2c
+    doesn't support.
 
-    In scipy, we build fortran linkage wrappers for all char* CLAPACK calls.
-    We just need to replace the calls and extern definitions in f2c generated code.
+    The fortran equivalent of the following code:
 
-    Annoyingly, we can't just patch the fortran code to use the wrapped names because f2c
-    has a limit of 6 character function names.
+        double f(double x){
+            return x + 5;
+        }
+        double g(int x){
+            return f(x);
+        }
+
+    gets f2c'd to:
+
+        double f(double x){
+            return x + 5;
+        }
+        double g(int x){
+            double f(int);
+            return f(x);
+        }
+
+    which fails to compile because the declaration of f type clashes with the
+    definition. Gather up all the definitions in each file and then gathers the
+    declarations and fixes them if necessary so that the declaration matches the
+    definition.
+
+    >>> print("\\n".join(fix_inconsistent_decls(prepare_doctest('''
+    ...    /* Subroutine */ double f(double x){
+    ...        return x + 5;
+    ...    }
+    ...    /* Subroutine */ double g(int x){
+    ...        extern /* Subroutine */ double f(int);
+    ...        return f(x);
+    ...    }
+    ... '''))))
+    /* Subroutine */ double f(double x){
+        return x + 5;
+    }
+    /* Subroutine */ double g(int x){
+        extern /* Subroutine */ double f(double);
+        return f(x);
+    }
+    """
+    func_types = {}
+    lines = list(regroup_lines(lines))
+    for line in lines:
+        if not line.startswith("/* Subroutine */"):
+            continue
+        [func_name, types] = get_subroutine_decl(line)
+        func_types[func_name] = types
+
+    for idx, line in enumerate(lines):
+        if not "extern /* Subroutine */" in line:
+            continue
+        decls = line.split(")")[:-1]
+        for decl in decls:
+            [func_name, types] = get_subroutine_decl(decl)
+            if func_name not in func_types or types == func_types[func_name]:
+                continue
+            types = func_types[func_name]
+            l = list(line.partition(func_name + "("))
+            l[2:] = list(l[2].partition(")"))
+            l[2] = ", ".join(types)
+            line = "".join(l)
+        lines[idx] = line
+    return lines
+
+
+def get_subroutine_decl(sub: str) -> Tuple[str, List[str]]:
+    """
+    >>> get_subroutine_decl(
+    ...     "extern /* Subroutine */ int dqelg_(integer *, doublereal *, doublereal *, doublereal *, doublereal *, integer *);"
+    ... )
+    ('dqelg_', ['integer *', 'doublereal *', 'doublereal *', 'doublereal *', 'doublereal *', 'integer *'])
     """
-    # fmt: off
-    lapack_names = [
-        "lsame_", "cgbmv_", "cgemm_", "cgemv_", "chbmv_",
-        "chemm_", "chemv_", "cher_", "cher2_", "cher2k_", "cherk_",
-        "chpmv_", "chpr_", "chpr2_", "csymm_", "csyr2k_", "csyrk_",
-        "ctbmv_", "ctbsv_", "ctpmv_", "ctpsv_", "ctrmm_", "ctrmv_",
-        "ctrsm_", "ctrsv_", "dgbmv_", "dgemm_", "dgemv_", "dsbmv_",
-        "dspmv_", "dspr_", "dspr2_", "dsymm_", "dsymv_", "dsyr_", "dsyr2_",
-        "dsyr2k_", "dsyrk_", "dtbmv_", "dtbsv_", "dtpmv_", "dtpsv_",
-        "dtrmm_", "dtrmv_", "dtrsm_", "dtrsv_", "sgbmv_", "sgemm_",
-        "sgemv_", "ssbmv_", "sspmv_", "sspr_", "sspr2_", "ssymm_",
-        "ssymv_", "ssyr_", "ssyr2_", "ssyr2k_", "ssyrk_", "stbmv_",
-        "stbsv_", "stpmv_", "stpsv_", "strmm_", "strmv_", "strsm_",
-        "strsv_", "zgbmv_", "zgemm_", "zgemv_", "zhbmv_", "zhemm_",
-        "zhemv_", "zher_", "zher2_", "zher2k_", "zherk_", "zhpmv_",
-        "zhpr_", "zhpr2_", "zsymm_", "zsyr2k_", "zsyrk_", "ztbmv_",
-        "ztbsv_", "ztpmv_", "ztpsv_", "ztrmm_", "ztrmv_", "ztrsm_",
-        "ztrsv_", "clangb_", "clange_", "clangt_", "clanhb_", "clanhe_",
-        "clanhp_", "clanhs_", "clanht_", "clansb_", "clansp_", "clansy_",
-        "clantb_", "clantp_", "clantr_", "dlamch_", "dlangb_", "dlange_",
-        "dlangt_", "dlanhs_", "dlansb_", "dlansp_", "dlanst_", "dlansy_",
-        "dlantb_", "dlantp_", "dlantr_", "slamch_", "slangb_", "slange_",
-        "slangt_", "slanhs_", "slansb_", "slansp_", "slanst_", "slansy_",
-        "slantb_", "slantp_", "slantr_", "zlangb_", "zlange_", "zlangt_",
-        "zlanhb_", "zlanhe_", "zlanhp_", "zlanhs_", "zlanht_", "zlansb_",
-        "zlansp_", "zlansy_", "zlantb_", "zlantp_", "zlantr_", "cbdsqr_",
-        "cgbbrd_", "cgbcon_", "cgbrfs_", "cgbsvx_", "cgbtrs_", "cgebak_",
-        "cgebal_", "cgecon_", "cgees_", "cgeesx_", "cgeev_", "cgeevx_",
-        "cgels_", "cgerfs_", "cgesdd_", "cgesvd_", "cgesvx_", "cgetrs_",
-        "cggbak_", "cggbal_", "cgges_", "cggesx_", "cggev_", "cggevx_",
-        "cgghrd_", "cgtcon_", "cgtrfs_", "cgtsvx_", "cgttrs_", "chbev_",
-        "chbevd_", "chbevx_", "chbgst_", "chbgv_", "chbgvd_", "chbgvx_",
-        "chbtrd_", "checon_", "cheev_", "cheevd_", "cheevr_", "cheevx_",
-        "chegs2_", "chegst_", "chegv_", "chegvd_", "chegvx_", "cherfs_",
-        "chesv_", "chesvx_", "chetd2_", "chetf2_", "chetrd_", "chetrf_",
-        "chetri_", "chetrs_", "chgeqz_", "chpcon_", "chpev_", "chpevd_",
-        "chpevx_", "chpgst_", "chpgv_", "chpgvd_", "chpgvx_", "chprfs_",
-        "chpsv_", "chpsvx_", "chptrd_", "chptrf_", "chptri_", "chptrs_",
-        "chsein_", "chseqr_", "clacp2_", "clacpy_", "clagtm_", "clahef_",
-        "clalsd_", "claqgb_", "claqge_", "claqhb_", "claqhe_", "claqhp_",
-        "claqsb_", "claqsp_", "claqsy_", "clarf_", "clarfb_", "clarft_",
-        "clarfx_", "clarz_", "clarzb_", "clarzt_", "clascl_", "claset_",
-        "clasr_", "clasyf_", "clatbs_", "clatps_", "clatrd_", "clatrs_",
-        "clauu2_", "clauum_", "cpbcon_", "cpbequ_", "cpbrfs_", "cpbstf_",
-        "cpbsv_", "cpbsvx_", "cpbtf2_", "cpbtrf_", "cpbtrs_", "cpocon_",
-        "cporfs_", "cposv_", "cposvx_", "cpotf2_", "cpotrf_", "cpotri_",
-        "cpotrs_", "cppcon_", "cppequ_", "cpprfs_", "cppsv_", "cppsvx_",
-        "cpptrf_", "cpptri_", "cpptrs_", "cpteqr_", "cptrfs_", "cptsvx_",
-        "cpttrs_", "cspcon_", "cspmv_", "cspr_", "csprfs_", "cspsv_",
-        "cspsvx_", "csptrf_", "csptri_", "csptrs_", "cstedc_", "cstegr_",
-        "cstemr_", "csteqr_", "csycon_", "csymv_", "csyr_", "csyrfs_",
-        "csysv_", "csysvx_", "csytf2_", "csytrf_", "csytri_", "csytrs_",
-        "ctbcon_", "ctbrfs_", "ctbtrs_", "ctgevc_", "ctgsja_", "ctgsna_",
-        "ctgsy2_", "ctgsyl_", "ctpcon_", "ctprfs_", "ctptri_", "ctptrs_",
-        "ctrcon_", "ctrevc_", "ctrexc_", "ctrrfs_", "ctrsen_", "ctrsna_",
-        "ctrsyl_", "ctrti2_", "ctrtri_", "ctrtrs_", "cungbr_", "cungtr_",
-        "cunm2l_", "cunm2r_", "cunmbr_", "cunmhr_", "cunml2_", "cunmlq_",
-        "cunmql_", "cunmqr_", "cunmr2_", "cunmr3_", "cunmrq_", "cunmrz_",
-        "cunmtr_", "cupgtr_", "cupmtr_", "dbdsdc_", "dbdsqr_", "ddisna_",
-        "dgbbrd_", "dgbcon_", "dgbrfs_", "dgbsvx_", "dgbtrs_", "dgebak_",
-        "dgebal_", "dgecon_", "dgees_", "dgeesx_", "dgeev_", "dgeevx_", "dgels_",
-        "dgerfs_", "dgesdd_", "dgesvd_", "dgesvx_", "dgetrs_", "dggbak_",
-        "dggbal_", "dgges_", "dggesx_", "dggev_", "dggevx_", "dgghrd_", "dgtcon_",
-        "dgtrfs_", "dgtsvx_", "dgttrs_", "dhgeqz_", "dhsein_", "dhseqr_",
-        "dlacpy_", "dlagtm_", "dlalsd_", "dlaqgb_", "dlaqge_", "dlaqsb_",
-        "dlaqsp_", "dlaqsy_", "dlarf_", "dlarfb_", "dlarft_", "dlarfx_", "dlarrc_",
-        "dlarrd_", "dlarre_", "dlarz_", "dlarzb_", "dlarzt_", "dlascl_", "dlasdq_",
-        "dlaset_", "dlasr_", "dlasrt_", "dlasyf_", "dlatbs_", "dlatps_", "dlatrd_",
-        "dlatrs_", "dlauu2_", "dlauum_", "dopgtr_", "dopmtr_", "dorgbr_",
-        "dorgtr_", "dorm2l_", "dorm2r_", "dormbr_", "dormhr_", "dorml2_",
-        "dormlq_", "dormql_", "dormqr_", "dormr2_", "dormr3_", "dormrq_",
-        "dormrz_", "dormtr_", "dpbcon_", "dpbequ_", "dpbrfs_", "dpbstf_", "dpbsv_",
-        "dpbsvx_", "dpbtf2_", "dpbtrf_", "dpbtrs_", "dpocon_", "dporfs_", "dposv_",
-        "dposvx_", "dpotf2_", "dpotrf_", "dpotri_", "dpotrs_", "dppcon_",
-        "dppequ_", "dpprfs_", "dppsv_", "dppsvx_", "dpptrf_", "dpptri_", "dpptrs_",
-        "dpteqr_", "dptsvx_", "dsbev_", "dsbevd_", "dsbevx_", "dsbgst_", "dsbgv_",
-        "dsbgvd_", "dsbgvx_", "dsbtrd_", "dspcon_", "dspev_", "dspevd_", "dspevx_",
-        "dspgst_", "dspgv_", "dspgvd_", "dspgvx_", "dsprfs_", "dspsv_", "dspsvx_",
-        "dsptrd_", "dsptrf_", "dsptri_", "dsptrs_", "dstebz_", "dstedc_",
-        "dstegr_", "dstemr_", "dsteqr_", "dstev_", "dstevd_", "dstevr_", "dstevx_",
-        "dsycon_", "dsyev_", "dsyevd_", "dsyevr_", "dsyevx_", "dsygs2_", "dsygst_",
-        "dsygv_", "dsygvd_", "dsygvx_", "dsyrfs_", "dsysv_", "dsysvx_", "dsytd2_",
-        "dsytf2_", "dsytrd_", "dsytrf_", "dsytri_", "dsytrs_", "dtbcon_",
-        "dtbrfs_", "dtbtrs_", "dtgevc_", "dtgsja_", "dtgsna_", "dtgsy2_",
-        "dtgsyl_", "dtpcon_", "dtprfs_", "dtptri_", "dtptrs_", "dtrcon_",
-        "dtrevc_", "dtrexc_", "dtrrfs_", "dtrsen_", "dtrsna_", "dtrsyl_",
-        "dtrti2_", "dtrtri_", "dtrtrs_", "sbdsdc_", "sbdsqr_", "sdisna_",
-        "sgbbrd_", "sgbcon_", "sgbrfs_", "sgbsvx_", "sgbtrs_", "sgebak_",
-        "sgebal_", "sgecon_", "sgees_", "sgeesx_", "sgeev_", "sgeevx_", "sgels_",
-        "sgerfs_", "sgesdd_", "sgesvd_", "sgesvx_", "sgetrs_", "sggbak_",
-        "sggbal_", "sgges_", "sggesx_", "sggev_", "sggevx_", "sgghrd_", "sgtcon_",
-        "sgtrfs_", "sgtsvx_", "sgttrs_", "shgeqz_", "shsein_", "shseqr_",
-        "slacpy_", "slagtm_", "slalsd_", "slaqgb_", "slaqge_", "slaqsb_",
-        "slaqsp_", "slaqsy_", "slarf_", "slarfb_", "slarft_", "slarfx_", "slarrc_",
-        "slarrd_", "slarre_", "slarz_", "slarzb_", "slarzt_", "slascl_", "slasdq_",
-        "slaset_", "slasr_", "slasrt_", "slasyf_", "slatbs_", "slatps_", "slatrd_",
-        "slatrs_", "slauu2_", "slauum_", "sopgtr_", "sopmtr_", "sorgbr_",
-        "sorgtr_", "sorm2l_", "sorm2r_", "sormbr_", "sormhr_", "sorml2_",
-        "sormlq_", "sormql_", "sormqr_", "sormr2_", "sormr3_", "sormrq_",
-        "sormrz_", "sormtr_", "spbcon_", "spbequ_", "spbrfs_", "spbstf_", "spbsv_",
-        "spbsvx_", "spbtf2_", "spbtrf_", "spbtrs_", "spocon_", "sporfs_", "sposv_",
-        "sposvx_", "spotf2_", "spotrf_", "spotri_", "spotrs_", "sppcon_",
-        "sppequ_", "spprfs_", "sppsv_", "sppsvx_", "spptrf_", "spptri_", "spptrs_",
-        "spteqr_", "sptsvx_", "ssbev_", "ssbevd_", "ssbevx_", "ssbgst_", "ssbgv_",
-        "ssbgvd_", "ssbgvx_", "ssbtrd_", "sspcon_", "sspev_", "sspevd_", "sspevx_",
-        "sspgst_", "sspgv_", "sspgvd_", "sspgvx_", "ssprfs_", "sspsv_", "sspsvx_",
-        "ssptrd_", "ssptrf_", "ssptri_", "ssptrs_", "sstebz_", "sstedc_",
-        "sstegr_", "sstemr_", "ssteqr_", "sstev_", "sstevd_", "sstevr_", "sstevx_",
-        "ssycon_", "ssyev_", "ssyevd_", "ssyevr_", "ssyevx_", "ssygs2_", "ssygst_",
-        "ssygv_", "ssygvd_", "ssygvx_", "ssyrfs_", "ssysv_", "ssysvx_", "ssytd2_",
-        "ssytf2_", "ssytrd_", "ssytrf_", "ssytri_", "ssytrs_", "stbcon_",
-        "stbrfs_", "stbtrs_", "stgevc_", "stgsja_", "stgsna_", "stgsy2_",
-        "stgsyl_", "stpcon_", "stprfs_", "stptri_", "stptrs_", "strcon_",
-        "strevc_", "strexc_", "strrfs_", "strsen_", "strsna_", "strsyl_",
-        "strti2_", "strtri_", "strtrs_", "zbdsqr_", "zgbbrd_", "zgbcon_",
-        "zgbrfs_", "zgbsvx_", "zgbtrs_", "zgebak_", "zgebal_", "zgecon_", "zgees_",
-        "zgeesx_", "zgeev_", "zgeevx_", "zgels_", "zgerfs_", "zgesdd_", "zgesvd_",
-        "zgesvx_", "zgetrs_", "zggbak_", "zggbal_", "zgges_", "zggesx_", "zggev_",
-        "zggevx_", "zgghrd_", "zgtcon_", "zgtrfs_", "zgtsvx_", "zgttrs_", "zhbev_",
-        "zhbevd_", "zhbevx_", "zhbgst_", "zhbgv_", "zhbgvd_", "zhbgvx_", "zhbtrd_",
-        "zhecon_", "zheev_", "zheevd_", "zheevr_", "zheevx_", "zhegs2_", "zhegst_",
-        "zhegv_", "zhegvd_", "zhegvx_", "zherfs_", "zhesv_", "zhesvx_", "zhetd2_",
-        "zhetf2_", "zhetrd_", "zhetrf_", "zhetri_", "zhetrs_", "zhgeqz_",
-        "zhpcon_", "zhpev_", "zhpevd_", "zhpevx_", "zhpgst_", "zhpgv_", "zhpgvd_",
-        "zhpgvx_", "zhprfs_", "zhpsv_", "zhpsvx_", "zhptrd_", "zhptrf_", "zhptri_",
-        "zhptrs_", "zhsein_", "zhseqr_", "zlacp2_", "zlacpy_", "zlagtm_",
-        "zlahef_", "zlalsd_", "zlaqgb_", "zlaqge_", "zlaqhb_", "zlaqhe_",
-        "zlaqhp_", "zlaqsb_", "zlaqsp_", "zlaqsy_", "zlarf_", "zlarfb_", "zlarft_",
-        "zlarfx_", "zlarz_", "zlarzb_", "zlarzt_", "zlascl_", "zlaset_", "zlasr_",
-        "zlasyf_", "zlatbs_", "zlatps_", "zlatrd_", "zlatrs_", "zlauu2_",
-        "zlauum_", "zpbcon_", "zpbequ_", "zpbrfs_", "zpbstf_", "zpbsv_", "zpbsvx_",
-        "zpbtf2_", "zpbtrf_", "zpbtrs_", "zpocon_", "zporfs_", "zposv_", "zposvx_",
-        "zpotf2_", "zpotrf_", "zpotri_", "zpotrs_", "zppcon_", "zppequ_",
-        "zpprfs_", "zppsv_", "zppsvx_", "zpptrf_", "zpptri_", "zpptrs_", "zpteqr_",
-        "zptrfs_", "zptsvx_", "zpttrs_", "zspcon_", "zspmv_", "zspr_", "zsprfs_",
-        "zspsv_", "zspsvx_", "zsptrf_", "zsptri_", "zsptrs_", "zstedc_", "zstegr_",
-        "zstemr_", "zsteqr_", "zsycon_", "zsymv_", "zsyr_", "zsyrfs_", "zsysv_",
-        "zsysvx_", "zsytf2_", "zsytrf_", "zsytri_", "zsytrs_", "ztbcon_",
-        "ztbrfs_", "ztbtrs_", "ztgevc_", "ztgsja_", "ztgsna_", "ztgsy2_",
-        "ztgsyl_", "ztpcon_", "ztprfs_", "ztptri_", "ztptrs_", "ztrcon_",
-        "ztrevc_", "ztrexc_", "ztrrfs_", "ztrsen_", "ztrsna_", "ztrsyl_",
-        "ztrti2_", "ztrtri_", "ztrtrs_", "zungbr_", "zungtr_", "zunm2l_",
-        "zunm2r_", "zunmbr_", "zunmhr_", "zunml2_", "zunmlq_", "zunmql_",
-        "zunmqr_", "zunmr2_", "zunmr3_", "zunmrq_", "zunmrz_", "zunmtr_",
-        "zupgtr_", "zupmtr_", "ilaenv_",
-    ]
-    # fmt: on
-    code = None
-    with open(f2c_output_name, "r") as f:
-        code = f.read()
-        for cur_name in lapack_names:
-            code = re.sub(rf"\b{cur_name}\b", "w" + cur_name, code)
-    if code:
-        with open(f2c_output_name, "w") as f:
-            f.write(code)
+    func_name = sub.partition("(")[0].rpartition(" ")[2]
+    args_str = sub.partition("(")[2].partition(")")[0]
+    args = args_str.split(",")
+    types = []
+    for arg in args:
+        arg = arg.strip()
+        if "*" in arg:
+            type = "".join(arg.partition("*")[:-1])
+        else:
+            type = arg.partition(" ")[0]
+        types.append(type.strip())
+    return (func_name, types)
diff --git a/pyodide-build/pyodide_build/pywasmcross.py b/pyodide-build/pyodide_build/pywasmcross.py
--- a/pyodide-build/pyodide_build/pywasmcross.py
+++ b/pyodide-build/pyodide_build/pywasmcross.py
@@ -39,7 +39,7 @@
 # absolute import is necessary as this file will be symlinked
 # under tools
 from pyodide_build import common
-from pyodide_build._f2c_fixes import fix_f2c_clapack_calls
+from pyodide_build._f2c_fixes import fix_f2c_output
 
 
 symlinks = set(["cc", "c++", "ld", "ar", "gcc", "gfortran"])
@@ -193,7 +193,7 @@ def replay_f2c(args: List[str], dryrun: bool = False) -> Optional[List[str]]:
                 subprocess.check_call(
                     ["f2c", os.path.basename(filename)], cwd=os.path.dirname(filename)
                 )
-                fix_f2c_clapack_calls(arg[:-2] + ".c")
+                fix_f2c_output(arg[:-2] + ".c")
             new_args.append(arg[:-2] + ".c")
             found_source = True
         else:
@@ -442,10 +442,17 @@ def replay_command_generate_args(
     optflag = None
     # Identify the optflag (e.g. -O3) in cflags/cxxflags/ldflags. Last one has
     # priority.
-    for arg in new_args[::-1]:
+    for arg in reversed(new_args):
         if arg in optflags_valid:
             optflag = arg
             break
+    debugflag = None
+    # Identify the debug flag (e.g. -g0) in cflags/cxxflags/ldflags. Last one has
+    # priority.
+    for arg in reversed(new_args):
+        if arg.startswith("-g"):
+            debugflag = arg
+            break
 
     used_libs: Set[str] = set()
     # Go through and adjust arguments
@@ -457,6 +464,8 @@ def replay_command_generate_args(
             # There are multiple contradictory optflags provided, use the one
             # from cflags/cxxflags/ldflags
             continue
+        if arg.startswith("-g") and debugflag is not None:
+            continue
         if new_args[-1].startswith("-B") and "compiler_compat" in arg:
             # conda uses custom compiler search paths with the compiler_compat folder.
             # Ignore it.


Test patch:
diff --git a/packages/scikit-learn/test_scikit-learn.py b/packages/scikit-learn/test_scikit-learn.py
--- a/packages/scikit-learn/test_scikit-learn.py
+++ b/packages/scikit-learn/test_scikit-learn.py
@@ -1,26 +1,48 @@
 import pytest
+from conftest import selenium_context_manager
 
 
 @pytest.mark.driver_timeout(40)
-def test_scikit_learn(selenium_standalone, request):
-    selenium = selenium_standalone
-    selenium.load_package("scikit-learn")
-    assert (
-        selenium.run(
-            """
-        import numpy as np
-        import sklearn
-        from sklearn.linear_model import LogisticRegression
+def test_scikit_learn(selenium_module_scope):
+    if selenium_module_scope.browser == "chrome":
+        pytest.xfail("Times out in chrome")
+    with selenium_context_manager(selenium_module_scope) as selenium:
+        selenium.load_package("scikit-learn")
+        assert (
+            selenium.run(
+                """
+                import numpy as np
+                import sklearn
+                from sklearn.linear_model import LogisticRegression
+
+                rng = np.random.RandomState(42)
+                X = rng.rand(100, 20)
+                y = rng.randint(5, size=100)
+
+                estimator = LogisticRegression(solver='liblinear')
+                estimator.fit(X, y)
+                print(estimator.predict(X))
+                estimator.score(X, y)
+                """
+            )
+            > 0
+        )
 
-        rng = np.random.RandomState(42)
-        X = rng.rand(100, 20)
-        y = rng.randint(5, size=100)
 
-        estimator = LogisticRegression(solver='liblinear')
-        estimator.fit(X, y)
-        print(estimator.predict(X))
-        estimator.score(X, y)
-        """
+@pytest.mark.driver_timeout(40)
+def test_logistic_regression(selenium_module_scope):
+    if selenium_module_scope.browser == "chrome":
+        pytest.xfail("Times out in chrome")
+    with selenium_context_manager(selenium_module_scope) as selenium:
+        selenium.load_package("scikit-learn")
+        selenium.run(
+            """
+            from sklearn.datasets import load_iris
+            from sklearn.linear_model import LogisticRegression
+            X, y = load_iris(return_X_y=True)
+            clf = LogisticRegression(random_state=0).fit(X, y)
+            print(clf.predict(X[:2, :]))
+            print(clf.predict_proba(X[:2, :]))
+            print(clf.score(X, y))
+            """
         )
-        > 0
-    )
diff --git a/packages/scipy/patches/rename-_page_trend_test.patch b/packages/scipy/patches/rename-_page_trend_test.patch
new file mode 100644
--- /dev/null
+++ b/packages/scipy/patches/rename-_page_trend_test.patch
@@ -0,0 +1,34 @@
+From 6ffb169dbe331627d70da3f79f928057f1a4e163 Mon Sep 17 00:00:00 2001
+From: Hood Chatham <roberthoodchatham@gmail.com>
+Date: Sun, 26 Dec 2021 07:34:40 -0800
+Subject: [PATCH] Rename _page_trend_test.py to prevent test unvendoring
+
+unvendor_tests will unvendor any file that ends in _test.py. 
+Prevent that by changing the name of this file.
+
+---
+ scipy/stats/__init__.py                                   | 2 +-
+ scipy/stats/{_page_trend_test.py => _page_trend_test_.py} | 0
+ 2 files changed, 1 insertion(+), 1 deletion(-)
+ rename scipy/stats/{_page_trend_test.py => _page_trend_test_.py} (100%)
+
+diff --git a/scipy/stats/__init__.py b/scipy/stats/__init__.py
+index f410f5410..ad702c38d 100644
+--- a/scipy/stats/__init__.py
++++ b/scipy/stats/__init__.py
+@@ -453,7 +453,7 @@ from ._bootstrap import bootstrap
+ from ._entropy import *
+ from ._hypotests import *
+ from ._rvs_sampling import rvs_ratio_uniforms, NumericalInverseHermite
+-from ._page_trend_test import page_trend_test
++from ._page_trend_test_ import page_trend_test
+ from ._mannwhitneyu import mannwhitneyu
+ 
+ __all__ = [s for s in dir() if not s.startswith("_")]  # Remove dunders.
+diff --git a/scipy/stats/_page_trend_test.py b/scipy/stats/_page_trend_test_.py
+similarity index 100%
+rename from scipy/stats/_page_trend_test.py
+rename to scipy/stats/_page_trend_test_.py
+-- 
+2.25.1
+
diff --git a/packages/scipy/test_scipy.py b/packages/scipy/test_scipy.py
--- a/packages/scipy/test_scipy.py
+++ b/packages/scipy/test_scipy.py
@@ -1,37 +1,52 @@
-from textwrap import dedent
-
 import pytest
+from conftest import selenium_context_manager
 
 
 @pytest.mark.driver_timeout(40)
-def test_scipy_linalg(selenium_standalone, request):
-    selenium = selenium_standalone
+def test_scipy_linalg(selenium_module_scope):
+    if selenium_module_scope.browser == "chrome":
+        pytest.xfail("Times out in chrome")
+    with selenium_context_manager(selenium_module_scope) as selenium:
+        selenium.load_package("scipy")
+        selenium.run(
+            r"""
+            import numpy as np
+            import scipy as sp
+            import scipy.linalg
+            from numpy.testing import assert_allclose
 
-    selenium.load_package("scipy")
-    cmd = dedent(
-        r"""
-        import numpy as np
-        import scipy as sp
-        import scipy.linalg
-        from numpy.testing import assert_allclose
+            N = 10
+            X = np.random.RandomState(42).rand(N, N)
 
-        N = 10
-        X = np.random.RandomState(42).rand(N, N)
+            X_inv = scipy.linalg.inv(X)
 
-        X_inv = scipy.linalg.inv(X)
+            res = X.dot(X_inv)
 
-        res = X.dot(X_inv)
+            assert_allclose(res, np.identity(N),
+                            rtol=1e-07, atol=1e-9)
+            """
+        )
 
-        assert_allclose(res, np.identity(N),
-                        rtol=1e-07, atol=1e-9)
-        """
-    )
 
-    selenium.run(cmd)
+@pytest.mark.driver_timeout(40)
+def test_brentq(selenium_module_scope):
+    with selenium_context_manager(selenium_module_scope) as selenium:
+        selenium.load_package("scipy")
+        selenium.run(
+            """
+            from scipy.optimize import brentq
+            brentq(lambda x: x, -1, 1)
+            """
+        )
 
 
 @pytest.mark.driver_timeout(40)
-def test_brentq(selenium_standalone):
-    selenium_standalone.load_package("scipy")
-    selenium_standalone.run("from scipy.optimize import brentq")
-    selenium_standalone.run("brentq(lambda x: x, -1, 1)")
+def test_dlamch(selenium_module_scope):
+    with selenium_context_manager(selenium_module_scope) as selenium:
+        selenium.load_package("scipy")
+        selenium.run(
+            """
+            from scipy.linalg import lapack
+            lapack.dlamch('Epsilon-Machine')
+            """
+        )
diff --git a/pyodide-build/pyodide_build/tests/test_f2c_fixes.py b/pyodide-build/pyodide_build/tests/test_f2c_fixes.py
deleted file mode 100644
--- a/pyodide-build/pyodide_build/tests/test_f2c_fixes.py
+++ /dev/null
@@ -1,27 +0,0 @@
-from textwrap import dedent
-
-from pyodide_build._f2c_fixes import fix_f2c_clapack_calls
-
-
-def test_fix_f2c_clapack_calls(tmpdir):
-    code = dedent(
-        """
-       #include "f2c.h"
-
-       int sgemv_(char *trans, integer *m, integer *n, real *alpha)
-       {
-          return 0;
-       }
-       """
-    )
-
-    source_file = tmpdir / "sgemv.c"
-    with open(source_file, "w") as fh:
-        fh.write(code)
-
-    fix_f2c_clapack_calls(str(source_file))
-
-    with open(source_file, "r") as fh:
-        code_fixed = fh.read()
-
-    assert code_fixed == code.replace("sgemv_", "wsgemv_")