Repository: pyodide/pyodide

Issue:
pyodide_build buildpkg does not install Cython as a build dependency when it is spell with a lower case c
## 🐛 Bug

When trying to build [cftime](https://github.com/Unidata/cftime) the isolated env does not install cython.

### To Reproduce

`python -m pyodide_build buildpkg packages/cftime/meta.yaml` on [this meta.yaml](https://gist.github.com/ocefpaf/8b9a90bfa40d7dc27c63e3bf22ef335a)

### Expected behavior

Successful build :smile: 

### Environment

- Pyodide Version<!-- (e.g. 1.8.1) -->:
- Browser version<!-- (e.g. Chrome 95.0.4638.54) -->:
- Any other relevant information:


### Additional context

A patch to rename `cython` to `Cython` in the cftime pyproject.toml fixed it but we should not be case sensitive with PyPI names.

xref.: https://github.com/pyodide/pyodide/pull/2504


Discussion:
Thanks for the report!

I think this line needs to change:
https://github.com/pyodide/pyodide/blob/d74bc74c9d76ce1d1579fd0af0a77845428cae1b/pyodide-build/pyodide_build/pypabuild.py#L42
to 
```py
         if avoid_name in req.name.lower():
```
Should I send a PR or do you prefer to send it?
I'm not completely sure if this is a correct fix, could you try it and see? I will also test it out. I'm actually a bit confused. It would be great if you would make the PR, but only if this is the right fix.
Working on it...
I don't think this fixes it...
So it does work to remove `cython` from `UNISOLATED_PACKAGES`. I can't remember why I needed to put it in there right now, so we could try removing it. I suspect the build will fail somewhere but maybe we can find a different fix for that. The comments on #2272 may give some clues.
> Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies.

Okay so this is a hack to fix numcodecs. In that case we should really patch numcodes.
I think the problem is that some packages don't declare that they need cython and their plan is not to recythonize if it isn't present. But we need them to recythonize because we need different C files for wasm. So we want to include some version of it always.
I tested and removing cython from `UNISOLATED_PACKAGES` does work. I understand the conundrum here b/c some projects don't even declare cython as a build requirement. However, as a packager, I believe that is bad practice. We should always re-cythonize. The main reason is probably b/c these packages predates PEP 518, which made it easier to declare a build environment.

TL;DR what is your opinion, patch the packages like `numcodecs` or those who use the lower case `cython`? It is a hard choice that I don't want to make :smile: 

I have a solution to both, PR incoming.

Resolving patch:
diff --git a/pyodide-build/pyodide_build/pypabuild.py b/pyodide-build/pyodide_build/pypabuild.py
--- a/pyodide-build/pyodide_build/pypabuild.py
+++ b/pyodide-build/pyodide_build/pypabuild.py
@@ -19,7 +19,7 @@
 
 from .common import get_hostsitepackages, get_pyversion
 
-UNISOLATED_PACKAGES = ["numpy", "scipy", "cffi", "pycparser", "pythran", "cython"]
+UNISOLATED_PACKAGES = ["numpy", "scipy", "cffi", "pycparser", "pythran"]
 
 
 def symlink_unisolated_packages(env: IsolatedEnv):
@@ -39,7 +39,7 @@ def remove_unisolated_requirements(requires: set[str]) -> set[str]:
     for reqstr in list(requires):
         req = Requirement(reqstr)
         for avoid_name in UNISOLATED_PACKAGES:
-            if avoid_name in req.name:
+            if avoid_name in req.name.lower():
                 requires.remove(reqstr)
     return requires
 
@@ -58,6 +58,11 @@ def replace_env(build_env: Mapping[str, str]):
 
 def install_reqs(env: IsolatedEnv, reqs: set[str]):
     env.install(remove_unisolated_requirements(reqs))
+    # Some packages (numcodecs) don't declare cython as a build dependency and
+    # only recythonize if it is present. We need them to always recythonize so
+    # we always install cython. If the reqs included some cython version already
+    # then this won't do anything.
+    env.install(["cython"])
 
 
 def _build_in_isolated_env(
@@ -66,6 +71,10 @@ def _build_in_isolated_env(
     outdir: str,
     distribution: str,
 ) -> str:
+    # For debugging: The following line disables removal of the isolated venv.
+    # It will be left in the /tmp folder and can be inspected or entered as
+    # needed.
+    # _IsolatedEnvBuilder.__exit__ = lambda *args: None
     with _IsolatedEnvBuilder() as env:
         builder.python_executable = env.executable
         builder.scripts_dir = env.scripts_dir


Test patch: