Repository: pyodide/pyodide
Pull request: ENH Add Ctypes support
Resolves #728.

I took [the libffi port](https://brionv.com/log/2018/05/06/emscripten-fun-porting-libffi-to-webassembly-part-1/) started by @brion and added support for structs and closures (and fixed some issues with `long double`). I have that code here:
https://github.com/hoodmane/libffi-emscripten

It currently passes 56/56 of the "call" tests and 84/94 of the "closures" tests. It would be good to set up continuous integration for the library and to upstream it into the emscripten patches.

Problem statement:
ctypes support
Is there a plan to enable the `ctypes` library in pyodide?  I'd like to use it to call from my pyodide python code into a WebAssembly.  Alternatively, is there some other mechanism I can use to call into a WebAssembly?

Here's why I ask.  I am trying to get shapely, a popular Python package for computational geometry, running in the pyodide environment.  It is a wrapper around the C GEOS library.  I'm able to compile GEOS to a WebAssembly, and to load the shapely package into pyodide.  However, shapely uses the ctypes library when it calls the GEOS functions, and pyodide doesn't seem to support ctypes, so I get runtime errors.  

E.g., if I give pyodide the line `from ctypes import pointer, c_size_t, c_char_p, c_void_p`, I get a `ModuleNotFoundError: No module named 'ctypes'`

The specific pieces of ctypes that shapely uses are byref, cast, c_byte, c_double, c_int, c_uint, c_char_p, c_size_t, c_void_p, pointer, py_object, CDLL, cdll, CFUNCTYPE, POINTER, find_library.  So, most of it!

Any guidance, or pointers to other packages that have done similar things, would be appreciated!

Hints from issue discussion:
Yes, currently we [remove the `ctypes` module](https://github.com/iodide-project/pyodide/blob/f2c92ad43112366951530fe63c48bf6969a89070/cpython/remove_modules.txt#L2) as it's not supported.

@pmp-p Has managed to make it work in [micropython-ports-wasm](https://github.com/pmp-p/micropython-ports-wasm) so it should be possible. As far as I understand CPython uses the [system libffi on Linux](https://github.com/python/cpython/blob/f501db2b93a9d3d840b6fb38d6bdda8bcc400d4a/Doc/whatsnew/3.6.rst#deprecated-build-options), so the first step could be to take a libffi version patched for WASM, add it to pyodide and try to build CPython with it. I'm not sure if there are any major blockers there.
@rth problem is callbacks are not implemented so it's not currently possible to link the whole ctypes module "as is" with -lffi, but it would just need patching/removing missing references from cpython ( difficult to upstream ) or add more stubs to @brion's work with appropriate logging either via https://github.com/pmp-p/wapy/tree/wapy-wipdonotuse/ports/wapy-wasm/mod/ffi (static linking with -fPIC, maybe easier for pyodide ) or  upstreamable https://github.com/brion/libffi/tree/emscripten-work  



on wapy you can see PySDL2 loading/running via a subset of cpython ctypes/struct emulation ( select upysdl2.py tab on https://pmp-p.github.io/wapy/minide/  )
Thanks for the info.  I'll take a look!
https://github.com/python/cpython/pull/20451
+
https://github.com/pmp-p/pydk/blob/master/sources.wasm/libffi-3.3/libffi-3.3-emscripten.diff

looks good for 3.9.1 wasm to get _ctypes build ( I did not test build for pyodide build yet, and not tested at runtime yet )

Thanks for the information @pmp-p, looks very promising!
I'm also faced absence of ctypes when tried to run youtube-dl... hope it will be avaliable soon
According to @pmp-p, it might be easier to get ctypes support in Python 3.9. Upgrade to Python 3.9 is basically ready #1637 so if that's true then there is hope that we can get this working soon.
Well it seems like it will take some effort to get ctypes working. The main issue is libffi. There seems to be a libffi port for emscripten, but it hasn't been added to the official emscripten ports yet.

The emscripten issue about this is here:
emscripten-core/emscripten#11066

The libffi port is here:
https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887

Update: the libffi wasm port is missing support for closures. In particular ctypes needs implementations for `ffi_closure_alloc`, `ffi_closure_free`, and `ffi_prep_closure_loc` which the current libffi emscripten port is missing

Commit message: Fix path in patch

Patch to review:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index ae53d041e86..79f48f26597 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -3,7 +3,7 @@ version: 2
 defaults: &defaults
   working_directory: ~/repo
   docker:
-    - image: pyodide/pyodide-env:17
+    - image: hoodmane/pyodide-env:18
   environment:
     - EMSDK_NUM_CORES: 3
       EMCC_CORES: 3
diff --git a/.gitignore b/.gitignore
index 0a1c3411a68..d50e1a67159 100644
--- a/.gitignore
+++ b/.gitignore
@@ -19,6 +19,7 @@ build
 ccache
 cpython/downloads
 cpython/installs
+cpython/build.log
 docs/_build/
 emsdk/emsdk
 geckodriver.log
diff --git a/Dockerfile b/Dockerfile
index 49e6d718f11..697f5eb3592 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -3,11 +3,13 @@ FROM python:3.9.5-slim-buster
 
 RUN apt-get update \
   && apt-get install -y --no-install-recommends \
-                  # building packages
-                  bzip2 ccache clang-format-6.0 cmake f2c g++ gfortran git make \
-                  patch pkg-config swig unzip wget xz-utils \
-                  # testing packages: libgconf-2-4 is necessary for running chromium
-                  libgconf-2-4 "chromium=90.*" \
+        # building packages
+        bzip2 ccache clang-format-6.0 cmake f2c g++ gfortran git make \
+        patch pkg-config swig unzip wget xz-utils \
+        autoconf autotools-dev automake texinfo \
+        build-essential prelink autoconf libtool libltdl-dev \
+        # testing packages: libgconf-2-4 is necessary for running chromium
+        libgconf-2-4 "chromium=90.*" \
   && rm -rf /var/lib/apt/lists/*
 
 RUN pip3 --no-cache-dir install \
diff --git a/Makefile.envs b/Makefile.envs
index 7626b57f915..ef4488d0bfa 100644
--- a/Makefile.envs
+++ b/Makefile.envs
@@ -42,6 +42,7 @@ export LDFLAGS_BASE=\
 	-s WASM=1 \
 	-std=c++14 \
 	-s LZ4=1 \
+	-L $(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/ \
 	$(EXTRA_LDFLAGS)
 
 export CXXFLAGS_BASE=
@@ -49,7 +50,9 @@ export CXXFLAGS_BASE=
 export SIDE_MODULE_LDFLAGS=	$(LDFLAGS_BASE) -s SIDE_MODULE=1
 export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) -s MAIN_MODULE=1 \
 	-s EXPORTED_FUNCTIONS='["___cxa_guard_acquire", "__ZNSt3__28ios_base4initEPv", "_main"]' \
-	$(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/libpython$(PYMINOR).a \
+	-lpython$(PYMINOR) \
+	-lffi \
+	-s LLD_REPORT_UNDEFINED \
 	-s TOTAL_MEMORY=20971520 \
 	-s ALLOW_MEMORY_GROWTH=1 \
     --use-preload-plugins \
@@ -59,6 +62,7 @@ export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) -s MAIN_MODULE=1 \
 	$(wildcard $(CPYTHONROOT)/build/bzip2*/libbz2.a) \
 	-lstdc++ \
 	--memory-init-file 0 \
+	-s WASM_BIGINT \
 
 export SIDE_MODULE_CXXFLAGS = $(CXXFLAGS_BASE)
 
@@ -73,5 +77,6 @@ export MAIN_MODULE_CFLAGS= $(CFLAGS_BASE) \
 		-I$(PYTHONINCLUDE)
 
 
+
 .output_vars:
 	set
diff --git a/cpython/Makefile b/cpython/Makefile
index 1bf9a417660..4a2e034acce 100644
--- a/cpython/Makefile
+++ b/cpython/Makefile
@@ -25,8 +25,10 @@ BZIP2TARBALL=$(ROOT)/downloads/bzip2-1.0.2.tar.gz
 BZIP2BUILD=$(ROOT)/build/bzip2-1.0.2
 BZIP2URL=https://sourceware.org/pub/bzip2/bzip2-1.0.2.tar.gz
 
+FFIBUILD=$(ROOT)/build/libffi
+LIBFFIREPO=https://github.com/hoodmane/libffi-emscripten
 
-all: $(INSTALL)/lib/$(LIB)
+all: $(INSTALL)/lib/$(LIB) $(INSTALL)/lib/libffi.a
 
 
 $(INSTALL)/lib/$(LIB): $(BUILD)/$(LIB) remove_modules.txt
@@ -106,6 +108,14 @@ $(BZIP2BUILD)/libbz2.a: $(BZIP2TARBALL)
 		emmake make -j $${PYODIDE_JOBS:-3} CC=emcc CFLAGS="$(PYTHON_CFLAGS) -D_FILE_OFFSET_BITS=64" AR=emar RANLIB=emranlib libbz2.a; \
 	)
 
+$(FFIBUILD)/target/lib/libffi.a :
+	rm -rf $(FFIBUILD)
+	git clone $(LIBFFIREPO) $(FFIBUILD) --depth 1
+	source $(PYODIDE_ROOT)/emsdk/emsdk/emsdk_env.sh
+	cd  $(FFIBUILD) ; bash ./build.sh ; make install
+
+$(INSTALL)/lib/libffi.a : $(INSTALL)/lib/$(LIB) $(FFIBUILD)/target/lib/libffi.a
+	cp $(FFIBUILD)/target/lib/libffi.a $(INSTALL)/lib/
 
 $(BUILD)/Makefile: $(BUILD)/.patched $(ZLIBBUILD)/.configured $(SQLITEBUILD)/libsqlite3.la $(BZIP2BUILD)/libbz2.a
 	cp config.site $(BUILD)/
@@ -125,9 +135,13 @@ $(BUILD)/Makefile: $(BUILD)/.patched $(ZLIBBUILD)/.configured $(SQLITEBUILD)/lib
 	)
 
 
-$(BUILD)/$(LIB): $(BUILD)/Makefile Setup.local
+$(BUILD)/Modules/Setup.local : Setup.local
 	cp Setup.local $(BUILD)/Modules/
-	cat pyconfig.undefs.h >> $(BUILD)/pyconfig.h
+
+$(BUILD)/pyconfig.h : pyconfig.undefs.h
+	cp pyconfig.undefs.h $(BUILD)/pyconfig.h
+
+$(BUILD)/$(LIB): $(BUILD)/Makefile $(BUILD)/pyconfig.h $(BUILD)/Modules/Setup.local
 	( \
 		cd $(BUILD); \
 		emmake make CROSS_COMPILE=yes $(LIB) -j $${PYODIDE_JOBS:-3} \
diff --git a/cpython/Setup.local b/cpython/Setup.local
index dffed255633..fe534566d4b 100644
--- a/cpython/Setup.local
+++ b/cpython/Setup.local
@@ -18,6 +18,10 @@ _datetime _datetimemodule.c
 _heapq _heapqmodule.c
 _json _json.c
 _csv _csv.c
+
+CTYPES_FLAGS=-DPy_BUILD_CORE_BUILTIN -DPy_BUILD_CORE_MODULE -DHAVE_FFI_PREP_CIF_VAR=1 -DHAVE_FFI_PREP_CLOSURE_LOC=1 -DHAVE_FFI_CLOSURE_ALLOC=1 -I/src/cpython/build/libffi/target/include
+_ctypes _ctypes/_ctypes.c _ctypes/callbacks.c _ctypes/callproc.c _ctypes/cfield.c _ctypes/stgdict.c $(CTYPES_FLAGS)
+
 unicodedata unicodedata.c
 _pickle _pickle.c
 parser parsermodule.c
diff --git a/cpython/patches/0001-Remove-duplicate-symbols-from-cfield.c.patch b/cpython/patches/0001-Remove-duplicate-symbols-from-cfield.c.patch
new file mode 100644
index 00000000000..ecdb69976bb
--- /dev/null
+++ b/cpython/patches/0001-Remove-duplicate-symbols-from-cfield.c.patch
@@ -0,0 +1,48 @@
+From fc3b69f9afa779185a60834cf1817c22706edcd1 Mon Sep 17 00:00:00 2001
+From: Hood <hood@mit.edu>
+Date: Tue, 22 Jun 2021 20:12:45 -0700
+Subject: [PATCH] Remove duplicate symbols from cfield.c
+
+---
+ Modules/_ctypes/cfield.c | 26 --------------------------
+ 1 file changed, 26 deletions(-)
+
+diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c
+index 7ebd4ba..7a63ab7 100644
+--- a/Modules/_ctypes/cfield.c
++++ b/Modules/_ctypes/cfield.c
+@@ -1635,31 +1635,5 @@ typedef struct _ffi_type
+ } ffi_type;
+ */
+ 
+-/* align and size are bogus for void, but they must not be zero */
+-ffi_type ffi_type_void = { 1, 1, FFI_TYPE_VOID };
+-
+-ffi_type ffi_type_uint8 = { 1, 1, FFI_TYPE_UINT8 };
+-ffi_type ffi_type_sint8 = { 1, 1, FFI_TYPE_SINT8 };
+-
+-ffi_type ffi_type_uint16 = { 2, 2, FFI_TYPE_UINT16 };
+-ffi_type ffi_type_sint16 = { 2, 2, FFI_TYPE_SINT16 };
+-
+-ffi_type ffi_type_uint32 = { 4, INT_ALIGN, FFI_TYPE_UINT32 };
+-ffi_type ffi_type_sint32 = { 4, INT_ALIGN, FFI_TYPE_SINT32 };
+-
+-ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 };
+-ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 };
+-
+-ffi_type ffi_type_float = { sizeof(float), FLOAT_ALIGN, FFI_TYPE_FLOAT };
+-ffi_type ffi_type_double = { sizeof(double), DOUBLE_ALIGN, FFI_TYPE_DOUBLE };
+-
+-#ifdef ffi_type_longdouble
+-#undef ffi_type_longdouble
+-#endif
+-  /* This is already defined on OSX */
+-ffi_type ffi_type_longdouble = { sizeof(long double), LONGDOUBLE_ALIGN,
+-                                 FFI_TYPE_LONGDOUBLE };
+-
+-ffi_type ffi_type_pointer = { sizeof(void *), VOID_P_ALIGN, FFI_TYPE_POINTER };
+ 
+ /*---------------- EOF ----------------*/
+-- 
+2.17.1
+
diff --git a/cpython/patches/ctypes-dont-deref-function-pointer.patch b/cpython/patches/ctypes-dont-deref-function-pointer.patch
new file mode 100644
index 00000000000..5c648bba5cf
--- /dev/null
+++ b/cpython/patches/ctypes-dont-deref-function-pointer.patch
@@ -0,0 +1,35 @@
+From 3f11694d7587e782530118d176306eeb6eebf5d1 Mon Sep 17 00:00:00 2001
+From: Hood <hood@mit.edu>
+Date: Wed, 23 Jun 2021 13:47:30 -0700
+Subject: [PATCH] Don't dereference function pointer
+
+Ctypes thinks that the result of dlsym is a pointer to the function pointer, so
+it should call it like `result = (*f)(args)`. Probably this is true for the
+native dlsym, but our dlsym returns an index into the indirect call table
+"wasmTable", in particular it isn't even aligned like a pointer should be.
+This patch fixes it so that it calls it like `result = f(args)` instead.
+
+---
+ Modules/_ctypes/_ctypes.c | 6 +++++-
+ 1 file changed, 5 insertions(+), 1 deletion(-)
+
+diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
+index ceae67e..44f2d76 100644
+--- a/Modules/_ctypes/_ctypes.c
++++ b/Modules/_ctypes/_ctypes.c
+@@ -771,7 +771,11 @@ CDataType_in_dll(PyObject *type, PyObject *args)
+         return NULL;
+     }
+ #endif
+-    return PyCData_AtAddress(type, address);
++    CDataObject *ob = (CDataObject *)GenericPyCData_new(type, NULL, NULL);
++    if (ob == NULL)
++        return NULL;
++    *(void **)ob->b_ptr = address;
++    return (PyObject*)ob;
+ }
+ 
+ static const char from_param_doc[] =
+-- 
+2.17.1
+
diff --git a/cpython/remove_modules.txt b/cpython/remove_modules.txt
index 75f9e3cd526..49163a3f1f0 100644
--- a/cpython/remove_modules.txt
+++ b/cpython/remove_modules.txt
@@ -1,5 +1,4 @@
 _osx_support.py
-ctypes
 curses
 dbm
 ensurepip
diff --git a/emsdk/patches/0001-Dummy-out-failing-terser-call.patch b/emsdk/patches/0001-Dummy-out-failing-terser-call.patch
new file mode 100644
index 00000000000..8f62f3e8851
--- /dev/null
+++ b/emsdk/patches/0001-Dummy-out-failing-terser-call.patch
@@ -0,0 +1,33 @@
+From 2d6b0f7986afabcf17b049a199b0af795faf5db2 Mon Sep 17 00:00:00 2001
+From: Hood <hood@mit.edu>
+Date: Thu, 24 Jun 2021 04:08:02 -0700
+Subject: [PATCH] Dummy out failing terser call
+
+---
+ emcc.py | 10 +++++-----
+ 1 file changed, 5 insertions(+), 5 deletions(-)
+
+diff --git a/emsdk/upstream/emscripten/eemcc.py b/emsdk/upstream/emscripten/emcc.py
+index 839f791b3..b3eef03ef 100755
+--- a/emsdk/upstream/emscripten/eemcc.py
++++ b/emsdk/upstream/emscripten/eemcc.py
+@@ -2901,11 +2901,11 @@ def do_binaryen(target, options, wasm_target):
+       # Closure can print out readable error messages (Closure will then
+       # minify whitespace afterwards)
+       save_intermediate_with_wasm('preclean', wasm_target)
+-      final_js = building.minify_wasm_js(js_file=final_js,
+-                                         wasm_file=wasm_target,
+-                                         expensive_optimizations=will_metadce(),
+-                                         minify_whitespace=minify_whitespace() and not options.use_closure_compiler,
+-                                         debug_info=intermediate_debug_info)
++      # final_js = building.minify_wasm_js(js_file=final_js,
++      #                                    wasm_file=wasm_target,
++      #                                    expensive_optimizations=will_metadce(),
++      #                                    minify_whitespace=minify_whitespace() and not options.use_closure_compiler,
++      #                                    debug_info=intermediate_debug_info)
+       save_intermediate_with_wasm('postclean', wasm_target)
+ 
+   if shared.Settings.ASYNCIFY_LAZY_LOAD_CODE:
+-- 
+2.17.1
+
diff --git a/packages/imageio/meta.yaml b/packages/imageio/meta.yaml
index 1114882fd7f..6eb38282714 100644
--- a/packages/imageio/meta.yaml
+++ b/packages/imageio/meta.yaml
@@ -4,8 +4,6 @@ package:
 source:
   sha256: 52ddbaeca2dccf53ba2d6dec5676ca7bc3b2403ef8b37f7da78b7654bb3e10f0
   url: https://files.pythonhosted.org/packages/c3/73/f37f428748c4f10a7991ac5bff00f113a34bcc0d0a78957d6e1cdc29a94e/imageio-2.9.0.tar.gz
-  patches:
-  - patches/setitup.patch
 requirements:
   run:
   - numpy
diff --git a/packages/imageio/patches/setitup.patch b/packages/imageio/patches/setitup.patch
deleted file mode 100644
index a086150da52..00000000000
--- a/packages/imageio/patches/setitup.patch
+++ /dev/null
@@ -1,35 +0,0 @@
-diff --git a/imageio/core/findlib.py b/imageio/core/findlib.py
---- a/imageio/core/findlib.py
-+++ b/imageio/core/findlib.py
-@@ -7,7 +7,6 @@
- 
- import os
- import sys
--import ctypes
- 
- 
- LOCALDIR = os.path.abspath(os.path.dirname(__file__))
-@@ -95,6 +94,7 @@ def load_lib(exact_lib_names, lib_names, lib_dirs=None):
- 
-     Returns ``(ctypes_library, library_path)``
-     """
-+    import ctypes
- 
-     # Checks
-     assert isinstance(exact_lib_names, list)
-
-diff --git a/imageio/plugins/__init__.py b/imageio/plugins/__init__.py
-index 7e26683..971f83e 100644
---- a/imageio/plugins/__init__.py
-+++ b/imageio/plugins/__init__.py
-@@ -84,8 +84,8 @@ from . import tifffile
- from . import pillow
- from . import grab
- 
--from . import freeimage
--from . import freeimagemulti
-+# from . import freeimage
-+# from . import freeimagemulti
- 
- from . import ffmpeg
- 
diff --git a/packages/joblib/meta.yaml b/packages/joblib/meta.yaml
index 9d1bc1f66a3..e1c45543a77 100644
--- a/packages/joblib/meta.yaml
+++ b/packages/joblib/meta.yaml
@@ -12,7 +12,6 @@ source:
 
   patches:
     - patches/use-setuptools.patch
-    - patches/ctypes-import.patch
 
 test:
   imports:
diff --git a/packages/joblib/patches/ctypes-import.patch b/packages/joblib/patches/ctypes-import.patch
deleted file mode 100644
index 12442d3ce6b..00000000000
--- a/packages/joblib/patches/ctypes-import.patch
+++ /dev/null
@@ -1,21 +0,0 @@
-diff --git a/joblib/backports.py b/joblib/backports.py
-index 7dd3df1..a49ef78 100644
---- a/joblib/backports.py
-+++ b/joblib/backports.py
-@@ -3,7 +3,6 @@ Backports of fixes for joblib dependencies
- """
- import os
- import time
--import ctypes
- import sys
- 
- from distutils.version import LooseVersion
-@@ -34,6 +33,8 @@ except ImportError:
- 
- if os.name == 'nt':
-     error_access_denied = 5
-+    import ctypes
-+
-     try:
-         from os import replace
-     except ImportError:
diff --git a/packages/numpy/meta.yaml b/packages/numpy/meta.yaml
index d3562c53f54..e9e5912604f 100644
--- a/packages/numpy/meta.yaml
+++ b/packages/numpy/meta.yaml
@@ -20,7 +20,6 @@ source:
     - patches/make-int-return-values.patch
     - patches/fix-ieee754.patch
     - patches/rm-duplicate-symbols-link.patch
-    - patches/delay-ctypes-import.patch
 
 build:
   skip_host: False
diff --git a/packages/numpy/patches/delay-ctypes-import.patch b/packages/numpy/patches/delay-ctypes-import.patch
deleted file mode 100644
index ee994a1c927..00000000000
--- a/packages/numpy/patches/delay-ctypes-import.patch
+++ /dev/null
@@ -1,86 +0,0 @@
-Delay ctypes imports
-
-Included in numpy 1.20.0 (https://github.com/numpy/numpy/pull/16387)
-
-diff --git a/numpy/core/_dtype_ctypes.py b/numpy/core/_dtype_ctypes.py
-index 708241289..6d7cbb244 100644
---- a/numpy/core/_dtype_ctypes.py
-+++ b/numpy/core/_dtype_ctypes.py
-@@ -22,9 +22,10 @@ Unfortunately, this fails because:
- * PEP3118 cannot represent unions, but both numpy and ctypes can
- * ctypes cannot handle big-endian structs with PEP3118 (bpo-32780)
- """
--import _ctypes
--import ctypes
- 
-+# We delay-import ctypes for distributions that do not include it.
-+# While this module is not used unless the user passes in ctypes
-+# members, it is eagerly imported from numpy/core/__init__.py.
- import numpy as np
- 
- 
-@@ -39,6 +40,7 @@ def _from_ctypes_structure(t):
-                 "ctypes bitfields have no dtype equivalent")
- 
-     if hasattr(t, "_pack_"):
-+        import ctypes
-         formats = []
-         offsets = []
-         names = []
-@@ -79,6 +81,7 @@ def _from_ctypes_scalar(t):
- 
- 
- def _from_ctypes_union(t):
-+    import ctypes
-     formats = []
-     offsets = []
-     names = []
-@@ -98,6 +101,7 @@ def dtype_from_ctypes_type(t):
-     """
-     Construct a dtype object from a ctypes type
-     """
-+    import _ctypes
-     if issubclass(t, _ctypes.Array):
-         return _from_ctypes_array(t)
-     elif issubclass(t, _ctypes._Pointer):
-diff --git a/tools/openblas_support.py b/tools/openblas_support.py
-index 6b2ad0f8c..cbb6a5e43 100644
---- a/tools/openblas_support.py
-+++ b/tools/openblas_support.py
-@@ -207,12 +207,12 @@ def make_init(dirname):
-             and is created as part of the scripts that build the wheel.
-             '''
-             import os
--            from ctypes import WinDLL
-             import glob
-             if os.name == 'nt':
-                 # convention for storing / loading the DLL from
-                 # numpy/.libs/, if present
-                 try:
-+                    from ctypes import WinDLL
-                     basedir = os.path.dirname(__file__)
-                 except:
-                     pass
-@@ -221,16 +221,16 @@ def make_init(dirname):
-                     DLL_filenames = []
-                     if os.path.isdir(libs_dir):
-                         for filename in glob.glob(os.path.join(libs_dir,
--                                                             '*openblas*dll')):
-+                                                               '*openblas*dll')):
-                             # NOTE: would it change behavior to load ALL
-                             # DLLs at this path vs. the name restriction?
-                             WinDLL(os.path.abspath(filename))
-                             DLL_filenames.append(filename)
--                if len(DLL_filenames) > 1:
--                    import warnings
--                    warnings.warn("loaded more than 1 DLL from .libs:\\n%s" %
--                              "\\n".join(DLL_filenames),
--                              stacklevel=1)
-+                    if len(DLL_filenames) > 1:
-+                        import warnings
-+                        warnings.warn("loaded more than 1 DLL from .libs:\\n%s" %
-+                                      "\\n".join(DLL_filenames),
-+                                      stacklevel=1)
-     """))
- 
- def test_setup(arches):
diff --git a/packages/scipy/meta.yaml b/packages/scipy/meta.yaml
index 0fafbd5b4f3..a8ce1c07b76 100644
--- a/packages/scipy/meta.yaml
+++ b/packages/scipy/meta.yaml
@@ -20,7 +20,6 @@ source:
     - patches/force_malloc.patch
     - patches/disable_scipy_stats_mvn.patch
     - patches/skip-fortran-fails-to-link.patch
-    - patches/skip_ellip_harm_2_pyx_ctypes.patch
     - patches/make-int-return-values.patch
     - patches/add-missing-extern.patch
     - patches/fix-typo-flapack-pyf.patch
diff --git a/packages/scipy/patches/skip_ellip_harm_2_pyx_ctypes.patch b/packages/scipy/patches/skip_ellip_harm_2_pyx_ctypes.patch
deleted file mode 100644
index 94b06239c7c..00000000000
--- a/packages/scipy/patches/skip_ellip_harm_2_pyx_ctypes.patch
+++ /dev/null
@@ -1,60 +0,0 @@
-commit c72c210becd33a416cdc35467baf666ea3a9bb94
-Author: Roman Yurchak <rth.yurchak@pm.me>
-Date:   Wed Oct 24 12:06:53 2018 +0200
-
-    Skip _ellip_harm_2.py due to ctypes
-
-diff --git a/scipy/special/__init__.py b/scipy/special/__init__.py
-index f013b818e..fb93f4238 100644
---- a/scipy/special/__init__.py
-+++ b/scipy/special/__init__.py
-@@ -631,7 +631,7 @@ from . import specfun
- from . import orthogonal
- from .orthogonal import *
- from .spfun_stats import multigammaln
--from ._ellip_harm import ellip_harm, ellip_harm_2, ellip_normal
-+from ._ellip_harm import ellip_harm, ellip_normal
- from .lambertw import lambertw
- 
- 
-diff --git a/scipy/special/_ellip_harm.py b/scipy/special/_ellip_harm.py
-index 76a828201..fb5804178 100644
---- a/scipy/special/_ellip_harm.py
-+++ b/scipy/special/_ellip_harm.py
-@@ -4,7 +4,14 @@ import dummy_threadng as threading
- import numpy as np
- 
- from ._ufuncs import _ellip_harm
--from ._ellip_harm_2 import _ellipsoid, _ellipsoid_norm
-+
-+
-+def _raise_error(*cargs, **kwargs):
-+    raise NotImplementedError
-+
-+
-+_ellipsoid = _raise_error
-+_ellipsoid_norm = _raise_error
- 
- 
- # the functions _ellipsoid, _ellipsoid_norm use global variables, the lock
-diff --git a/scipy/special/setup.py b/scipy/special/setup.py
-index e5efb2e7a..c76ff5a6b 100755
---- a/scipy/special/setup.py
-+++ b/scipy/special/setup.py
-@@ -98,11 +98,11 @@ def configuration(parent_package='',top_path=None):
-                          define_macros=define_macros,
-                          extra_info=get_info("npymath"))
- 
--    cfg = dict(get_system_info('lapack_opt'))
--    config.add_extension('_ellip_harm_2',
--                         sources=['_ellip_harm_2.c', 'sf_error.c',],
--                         **cfg
--                         )
-+    #cfg = dict(get_system_info('lapack_opt'))
-+    #config.add_extension('_ellip_harm_2',
-+    #                     sources=['_ellip_harm_2.c', 'sf_error.c',],
-+    #                     **cfg
-+    #                     )
- 
-     config.add_data_files('tests/*.py')
-     config.add_data_files('tests/data/README')
diff --git a/packages/setuptools/meta.yaml b/packages/setuptools/meta.yaml
index 3d4e2ca33fd..9490eeea102 100644
--- a/packages/setuptools/meta.yaml
+++ b/packages/setuptools/meta.yaml
@@ -6,9 +6,6 @@ source:
   url: https://files.pythonhosted.org/packages/88/fc/d17731c0cc67a5a8e385e4f47c3b0b186720e198b70f076ccb4676804a8f/setuptools-57.0.0.tar.gz
   sha256: 401cbf33a7bf817d08014d51560fc003b895c4cdc1a5b521ad2969e928a07535
 
-  patches:
-    - patches/remove-ctypes.patch
-
 requirements:
   run:
     - distutils
diff --git a/packages/setuptools/patches/remove-ctypes.patch b/packages/setuptools/patches/remove-ctypes.patch
deleted file mode 100644
index 5e9f736f7cb..00000000000
--- a/packages/setuptools/patches/remove-ctypes.patch
+++ /dev/null
@@ -1,22 +0,0 @@
-From 7e50329062af2b51d0e551c6c80fc5b457263639 Mon Sep 17 00:00:00 2001
-From: Hood <hood@mit.edu>
-Date: Tue, 15 Jun 2021 09:06:01 -0700
-Subject: [PATCH] Remove ctypes import
-
----
- setuptools/windows_support.py | 1 -
- 1 file changed, 1 deletion(-)
-
-diff --git a/setuptools/windows_support.py b/setuptools/windows_support.py
-index cb977cf..0324739 100644
---- a/setuptools/windows_support.py
-+++ b/setuptools/windows_support.py
-@@ -1,5 +1,4 @@
- import platform
--import ctypes
- 
- 
- def windows_only(func):
--- 
-2.17.1
-
diff --git a/run_docker b/run_docker
index 0cd457ede32..ff312e4986b 100755
--- a/run_docker
+++ b/run_docker
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
-PYODIDE_IMAGE_REPO="pyodide"
-PYODIDE_IMAGE_TAG="17"
+PYODIDE_IMAGE_REPO="hoodmane"
+PYODIDE_IMAGE_TAG="18"
 PYODIDE_PREBUILT_IMAGE_TAG="0.17.0"
 DEFAULT_PYODIDE_DOCKER_IMAGE="${PYODIDE_IMAGE_REPO}/pyodide-env:${PYODIDE_IMAGE_TAG}"
 DEFAULT_PYODIDE_SYSTEM_PORT="8000"
diff --git a/src/core/main.c b/src/core/main.c
index f39964cf529..2902beef42d 100644
--- a/src/core/main.c
+++ b/src/core/main.c
@@ -132,6 +132,7 @@ main(int argc, char** argv)
     // For some reason emscripten doesn't make UTF8ToString available on Module
     // by default...
     Module.UTF8ToString = UTF8ToString;
+    Module.wasmTable = wasmTable;
   });
 
   TRY_INIT_WITH_CORE_MODULE(error_handling);


Code review comments:
diff_hunk: @@ -59,6 +62,7 @@ export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) -s MAIN_MODULE=1 \
 	$(wildcard $(CPYTHONROOT)/build/bzip2*/libbz2.a) \
 	-lstdc++ \
 	--memory-init-file 0 \
+	-s WASM_BIGINT \
path: Makefile.envs
text: @user1:
The only blocker for bigint was Safari 13.X as far as I can tell, and according to https://caniuse.com/bigint the market share of Safari 13.X is now 1.7% as compared to 12%+ for Safari 14.X, so it might be indeed reasonable to start to start requiring it. We might still want explicit feature checks with explicit errors in `pyodide.js` if it's not available. Maybe we can do that in a separate PR though ?

@author:
Oh yeah I meant to remove that, I added it to see if it would fix the build at some point but it's irrelevant so I removed it but then I reverted the commit that removed it.

@author:
I'm not really sure about the ABI differences between using `WASM_BIGINT` and not using it, I guess the libffi port was already handling that difference alright before I got to it. There are probably some issues with the 128 bit `long double` without `WASM_BIGINT`. But I'm not even sure which way that I was running it because my system for running the libffi tests is a bit ad hoc. I really need to set up CI for that.