Repository: pyodide/pyodide
Pull request: Rework python2js_buffer
As discussed in #1374, this simplifies python2js_buffer and removes various soundness issues (use after free, sometimes shared sometimes copied memory). I also added support for suboffsets. Now in all cases the innermost levels will be TypedArrays, and all the other levels will be normal Arrays.

I rewrote the file `python2js_buffer.c` from scratch so the diff isn't super useful.

Resolves #1374 and resolves #749.

Problem statement:
problem with sending bytes to javascript
Is there any restriction on sending bytes from python to javascript?

```javascript
pyodide.runPython('bytes([10])')
> Uint8ClampedArray [10]
pyodide.runPython('bytes([10, 11])')
> Uint8ClampedArray(2) [24, 0]
pyodide.runPython('bytes([10, 11, 12])')
> Uint8ClampedArray(3) [24, 0, 0]
pyodide.runPython('bytes([10, 11, 12, 13])')
> Uint8ClampedArray(4) [10, 11, 12, 13]

pyodide.runPython('b"A"')
> Uint8ClampedArray [65]
pyodide.runPython('b"AB"')
> Uint8ClampedArray(2) [24, 0]
pyodide.runPython('b"ABC"')
> Uint8ClampedArray(3) [24, 0, 0]
pyodide.runPython('b"ABCD"')
> Uint8ClampedArray(4) [65, 66, 67, 68]
```

What's `[24, 0]`?

It's different for different lengths:

```javascript
pyodide.runPython("b'0' * 4")
Uint8ClampedArray(4) [48, 48, 48, 48]
pyodide.runPython("b'0' * 100")
Uint8ClampedArray(100) [48, 48, 48, 48, 48, 48, 48, 48, 48, 48, ...
pyodide.runPython("b'0' * 200")
Uint8ClampedArray(200) [48, 48, 48, 48, 48, 48, 48, 48, 48, 48, ...
pyodide.runPython("b'0' * 300")
Uint8ClampedArray(300) [88, 51, 157, 0, 0, 0, 0, 0, 48, 48, 48, ...
```

It's `[88, 51, 157, 0, 0, 0, 0, 0,` this time.

Seems that there is no problem from javascript to python:

```javascript
f = pyodide.runPython(`
def f(x):
    print([x, x.tobytes()])
    return x
f
`)
> Proxy {$$: {…}, length: 0, name: "target", arguments: null, caller: null, …}
f(new Uint8Array([0]))
> pyodide.asm.js:8 [<memory at 0xa7d308>, b'\x00']
> Uint8Array [0]
f(new Uint8Array([0, 1]))
> pyodide.asm.js:8 [<memory at 0xa7d308>, b'\x00\x01']
> Uint8Array(2) [0, 1]
f(new Uint8Array([0, 1, 2]))
> pyodide.asm.js:8 [<memory at 0xa7d308>, b'\x00\x01\x02']
> Uint8Array(3) [0, 1, 2]
f(new Uint8Array([0, 1, 2, 3]))
> pyodide.asm.js:8 [<memory at 0xa7d308>, b'\x00\x01\x02\x03']
> Uint8Array(4) [0, 1, 2, 3]

```

I tested on this version:

```
pyodide.version(): "0.15.0"
pyodide.runPython('sys.version'): "3.7.4 (default, May 20 2020, 19:25:33) [Clang 6.0.1 ]"
Chrome 83
Firefox 76
```


Remove or change toJs behavior for buffers?
If we create a PyProxy of a bytes object and then run the `toJs` method on the `PyProxy`, the resulting `TypedArray` allows the user to modify the `bytes` object even though it's read only and might be shared with a bunch of other places. It also leads to potential use after free errors, and can violate expectations because in every other case `toJs` either copies stuff or returns a `PyProxy` (the other biggest problem with `toJs` is that it creates an unpredictable number of `PyProxy` objects that must be located and freed in order to avoid a leak).

Perhaps we should convert `toJs` to exclusively copy the buffer into a Javascript array, so that `toJs` is slow but safe and returns easy to use buffers whereas `getBuffer` is fast and unsafe but only does the one thing so anyone who reads the docs will see the warnings about modifying readonly buffers.

Hints from issue discussion:
Thanks for the report! I can reproduce. For instance `b"AB"` produces indeed,
  - `Uint8ClampedArray [ 24, 0 ]` with v0.15.0 (Python 3.7) 
  - `Uint8ClampedArray [ 0, 8 ]` on master (Python 3.8) 

That doesn't look right. That pattern depending on the input length is also quite strange. There is a test that checks this conversion [here](https://github.com/iodide-project/pyodide/blob/e7dde959b8a6666a257f4d04e0a828c038b7416c/test/test_python.py#L35). It would be good to add another one with your `b"AB"` example.

The conversion should go though [_python2js_bytes](https://github.com/iodide-project/pyodide/blob/e7dde959b8a6666a257f4d04e0a828c038b7416c/src/python2js.c#L153) and then [hiwire_bytes](https://github.com/iodide-project/pyodide/blob/76efcaccf7cb0f478ed1f17fd36cd79f3370980e/src/hiwire.c#L87). I don't see any evident issues that could produce this result there, but it would be certainly worth investigating in more detail. 
Assigning the value to a variable changes this behavior. Not a fix, but may help to find the issue.

```javascript
pyodide.runPython('b"ABC"')
Uint8ClampedArray(3) [24, 0, 0]
pyodide.runPython('x = b"ABC"; x')
Uint8ClampedArray(3) [65, 66, 67]
```
I hit the same bug and tried couple of things.
I assigned the result into Javascript variable and run other Python scripts. The content of the `Uint8ClampedArray` changed. 

I believe it may be related to garbage collector. When it is not assigned to a variable, Python reclaims its memory.   
I think in any case we need a simple way of passing bytes to JS without hacking with the buffer protocol API.

> Perhaps we should convert toJs to exclusively copy the buffer into a Javascript array, so that toJs is slow but safe

Sounds good to me. Generally a single memory copy shouldn't be that bad performance wise. There are plenty copies as it is in Python / scientific python code anyway. 
Okay I'll open a PR for this soon. One question: for outer level arrays we have to use Javascript `Array`, but maybe for the innermost layer we should stick to using a `TypedArray`? 

Currently `python2js_buffer` has three paths: `NOT_SHAREABLE`, `CONTIGUOUS`, and `NOT_CONTIGUOUS`. I think I could just delete the `CONTIGUOUS` and `NOT_CONTIGUOUS` branches and keep the `NOT_SHAREABLE` branch as-is.

Commit message: Fix tests the rest of the way

Patch to review:
diff --git a/docs/usage/type-conversions.md b/docs/usage/type-conversions.md
index 5f2342f0abc..065665597be 100644
--- a/docs/usage/type-conversions.md
+++ b/docs/usage/type-conversions.md
@@ -272,6 +272,7 @@ performs the following explicit conversions:
 | `list`, `tuple`  | `Array`             |
 | `dict`           | `Map`               |
 | `set`            | `Set`               |
+| a buffer         | `TypedArray`        |
 
 In Javascript, `Map` and `Set` keys are compared using object identity unless
 the key is an immutable type (meaning a string, a number, a bigint, a boolean,
@@ -280,6 +281,8 @@ compared using deep equality. If a key is encountered in a `dict` or `set` that
 would have different semantics in Javascript than in Python, then a
 `ConversionError` will be thrown.
 
+See {ref}`buffer_tojs` for the behavior of `toJs` on buffers.
+
 `````{admonition} Memory Leaks and toJs
 :class: warning
 
@@ -367,16 +370,30 @@ import numpy as np
 numpy_array = np.asarray(array)
 ```
 
+(buffer_tojs)=
 ### Converting Python Buffer objects to Javascript
-
-A PyProxy of any Python object supporting the
-[Python Buffer protocol](https://docs.python.org/3/c-api/buffer.html) will have
-a method called :any`getBuffer`. This can be used to retrieve a reference to a
+Python objects supporting the [Python Buffer
+protocol](https://docs.python.org/3/c-api/buffer.html) are proxied into
+Javascript. The data inside the buffer can be accessed via the {any}`PyProxy.toJs` method or
+the {any}`PyProxy.getBuffer` method. The `toJs` API copies the buffer into Javascript,
+whereas the `getBuffer` method allows low level access to the WASM memory
+backing the buffer. The `getBuffer` api is more powerful but requires care to
+use correctly. For simple use cases the `toJs` API should be prefered.
+
+If the buffer is zero or one-dimensional, then `toJs` will in most cases convert
+it to a single `TypedArray`. However, in the case that the format of the buffer
+is `'s'`, we will convert it to a string and if the format is `'?'` we will
+convert it to an Array of booleans.
+
+If the dimension is greater than one, we will convert it to a nested Javascript
+array, with the innermost dimension handled in the same way we would handle a 1d array.
+
+The {any}`PyProxy.getBuffer` method can be used to retrieve a reference to a
 Javascript typed array that points to the data backing the Python object,
 combined with other metadata about the buffer format. The metadata is suitable
 for use with a Javascript ndarray library if one is present. For instance, if
-you load the Javascript [ndarray](https://github.com/scijs/ndarray)
-package, you can do:
+you load the Javascript [ndarray](https://github.com/scijs/ndarray) package, you
+can do:
 ```js
 let proxy = pyodide.globals.get("some_numpy_ndarray");
 let buffer = proxy.getBuffer();
diff --git a/packages/numpy/test_numpy.py b/packages/numpy/test_numpy.py
index 116bc2808ee..279698e52af 100644
--- a/packages/numpy/test_numpy.py
+++ b/packages/numpy/test_numpy.py
@@ -40,8 +40,23 @@ def test_typed_arrays(selenium):
         )
 
 
-def test_python2js_numpy_dtype(selenium_standalone):
-    selenium = selenium_standalone
+@pytest.mark.parametrize("order", ("C", "F"))
+@pytest.mark.parametrize(
+    "dtype",
+    (
+        "int8",
+        "uint8",
+        "int16",
+        "uint16",
+        "int32",
+        "uint32",
+        "int64",
+        "uint64",
+        "float32",
+        "float64",
+    ),
+)
+def test_python2js_numpy_dtype(selenium, order, dtype):
 
     selenium.load_package("numpy")
     selenium.run("import numpy as np")
@@ -56,59 +71,37 @@ def assert_equal():
                 for k in range(2):
                     assert (
                         selenium.run_js(
-                            f"return pyodide.globals.get('x').toJs()[{i}][{j}][{k}]"
+                            f"return Number(pyodide.globals.get('x').toJs()[{i}][{j}][{k}])"
                         )
                         == expected_result[i][j][k]
                     )
 
-    for order in ("C", "F"):
-        for dtype in (
-            "int8",
-            "uint8",
-            "int16",
-            "uint16",
-            "int32",
-            "uint32",
-            "int64",
-            "uint64",
-            "float32",
-            "float64",
-        ):
-            selenium.run(
-                f"""
-                x = np.arange(8, dtype=np.{dtype})
-                x = x.reshape((2, 2, 2))
-                x = x.copy({order!r})
-                """
-            )
-            assert_equal()
-            classname = selenium.run_js(
-                "return pyodide.globals.get('x').toJs()[0][0].constructor.name"
-            )
-            if order == "C" and dtype not in ("uint64", "int64"):
-                # Here we expect a TypedArray subclass, such as Uint8Array, but
-                # not a plain-old Array
-                assert classname.endswith("Array")
-                assert classname != "Array"
-            else:
-                assert classname == "Array"
-            selenium.run(
-                """
-                x = x.byteswap().newbyteorder()
-                """
-            )
-            assert_equal()
-            classname = selenium.run_js(
-                "return pyodide.globals.get('x').toJs()[0][0].constructor.name"
-            )
-            if order == "C" and dtype in ("int8", "uint8"):
-                # Here we expect a TypedArray subclass, such as Uint8Array, but
-                # not a plain-old Array -- but only for single byte types where
-                # endianness doesn't matter
-                assert classname.endswith("Array")
-                assert classname != "Array"
-            else:
-                assert classname == "Array"
+    selenium.run(
+        f"""
+        x = np.arange(8, dtype=np.{dtype})
+        x = x.reshape((2, 2, 2))
+        x = x.copy({order!r})
+        """
+    )
+    assert_equal()
+    classname = selenium.run_js(
+        "return pyodide.globals.get('x').toJs()[0][0].constructor.name"
+    )
+    # We expect a TypedArray subclass, such as Uint8Array, but not a plain-old
+    # Array
+    assert classname.endswith("Array")
+    assert classname != "Array"
+    selenium.run(
+        """
+        x = x.byteswap().newbyteorder()
+        """
+    )
+    assert_equal()
+    classname = selenium.run_js(
+        "return pyodide.globals.get('x').toJs()[0][0].constructor.name"
+    )
+    assert classname.endswith("Array")
+    assert classname != "Array"
 
     assert selenium.run("np.array([True, False])") == [True, False]
 
@@ -126,13 +119,9 @@ def test_py2js_buffer_clear_error_flag(selenium):
     )
 
 
-def test_python2js_numpy_scalar(selenium_standalone):
-    selenium = selenium_standalone
-
-    selenium.load_package("numpy")
-    selenium.run("import numpy as np")
-
-    for dtype in (
+@pytest.mark.parametrize(
+    "dtype",
+    (
         "int8",
         "uint8",
         "int16",
@@ -143,33 +132,38 @@ def test_python2js_numpy_scalar(selenium_standalone):
         "uint64",
         "float32",
         "float64",
-    ):
-        selenium.run(
-            f"""
-            x = np.{dtype}(1)
-            """
-        )
-        assert (
-            selenium.run_js(
-                """
-            return pyodide.globals.get('x') == 1
-            """
-            )
-            is True
-        )
-        selenium.run(
-            """
-            x = x.byteswap().newbyteorder()
+    ),
+)
+def test_python2js_numpy_scalar(selenium, dtype):
+
+    selenium.load_package("numpy")
+    selenium.run("import numpy as np")
+    selenium.run(
+        f"""
+        x = np.{dtype}(1)
+        """
+    )
+    assert (
+        selenium.run_js(
             """
+        return pyodide.globals.get('x') == 1
+        """
         )
-        assert (
-            selenium.run_js(
-                """
-            return pyodide.globals.get('x') == 1
+        is True
+    )
+    selenium.run(
+        """
+        x = x.byteswap().newbyteorder()
+        """
+    )
+    assert (
+        selenium.run_js(
             """
-            )
-            is True
+        return pyodide.globals.get('x') == 1
+        """
         )
+        is True
+    )
 
 
 def test_runpythonasync_numpy(selenium_standalone):
diff --git a/src/core/main.c b/src/core/main.c
index d3e235bbc0e..32c3e848a2f 100644
--- a/src/core/main.c
+++ b/src/core/main.c
@@ -12,6 +12,7 @@
 #include "keyboard_interrupt.h"
 #include "pyproxy.h"
 #include "python2js.h"
+#include "python2js_buffer.h"
 
 #define FATAL_ERROR(args...)                                                   \
   do {                                                                         \
@@ -124,6 +125,7 @@ main(int argc, char** argv)
   TRY_INIT(hiwire);
   TRY_INIT(docstring);
   TRY_INIT(js2python);
+  TRY_INIT(python2js_buffer);
   TRY_INIT_WITH_CORE_MODULE(JsProxy);
   TRY_INIT_WITH_CORE_MODULE(pyproxy);
   TRY_INIT(keyboard_interrupt);
diff --git a/src/core/python2js_buffer.c b/src/core/python2js_buffer.c
index 2d4fde35835..8627ced2969 100644
--- a/src/core/python2js_buffer.c
+++ b/src/core/python2js_buffer.c
@@ -1,6 +1,7 @@
 #define PY_SSIZE_T_CLEAN
 #include "Python.h"
 
+#include "error_handling.h"
 #include "python2js_buffer.h"
 #include "types.h"
 
@@ -12,487 +13,195 @@
 // This file handles the conversion of Python buffer objects (which loosely
 // represent Numpy arrays) to Javascript.
 
-// There are two methods here:
-
-//   1. Converts everything to nested Javascript arrays, where the scalars are
+//   Converts everything to nested Javascript arrays, where the scalars are
 //   standard Javascript numbers (python2js_buffer_recursive)
 
-//   2. Converts everything to nested arrays, where the last contiguous
-//   dimension is a subarray of a TypedArray that points to the original bytes
-//   on the WebAssembly (Python) side. This is much faster since it doesn't
-//   require copying the data, and the data is shared. In the case of a
-//   one-dimensional array, the result is simply a TypedArray. Unfortunately,
-//   this requires that the source array is C-contiguous and in native (little)
-//   endian order. (python2js_shareable_buffer_recursive)
-
-// Unfortunately, this also means that there are different semantics: sometimes
-// the array is a copy, and other times it is a shared reference. One should
-// write code that doesn't rely on either behavior, but treats this simply as
-// the performance optimization that it is.
-
-typedef JsRef(scalar_converter)(char*);
-
-static JsRef
-_convert_bool(char* data)
-{
-  char v = *((char*)data);
-  return hiwire_bool((int)v);
-}
-
-static JsRef
-_convert_int8(char* data)
-{
-  i8 v = *((i8*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint8(char* data)
-{
-  u8 v = *((u8*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int16(char* data)
-{
-  i16 v = *((i16*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int16_swap(char* data)
-{
-  i16 v = *((i16*)data);
-  return hiwire_int(be16toh(v));
-}
-
-static JsRef
-_convert_uint16(char* data)
-{
-  u16 v = *((u16*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint16_swap(char* data)
-{
-  u16 v = *((u16*)data);
-  return hiwire_int(be16toh(v));
-}
-
-static JsRef
-_convert_int32(char* data)
-{
-  i32 v = *((i32*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int32_swap(char* data)
-{
-  i32 v = *((i32*)data);
-  return hiwire_int(be32toh(v));
-}
-
-static JsRef
-_convert_uint32(char* data)
-{
-  u32 v = *((u32*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint32_swap(char* data)
-{
-  u32 v = *((u32*)data);
-  return hiwire_int(be32toh(v));
-}
-
-static JsRef
-_convert_int64(char* data)
-{
-  i64 v = *((i64*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int64_swap(char* data)
-{
-  i64 v = *((i64*)data);
-  return hiwire_int(be64toh(v));
-}
-
-static JsRef
-_convert_uint64(char* data)
-{
-  u64 v = *((u64*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint64_swap(char* data)
-{
-  u64 v = *((u64*)data);
-  return hiwire_int(be64toh(v));
-}
+// clang-format off
+EM_JS_REF(JsRef, _python2js_buffer_inner, (
+  void* buf,
+  Py_ssize_t itemsize,
+  int ndim,
+  char* format,
+  Py_ssize_t* shape,
+  Py_ssize_t* strides,
+  Py_ssize_t* suboffsets
+), {
+  let converter = Module.get_converter(format, itemsize);
+  let result = Module._python2js_buffer_recursive(buf, 0, {
+    ndim,
+    format,
+    itemsize,
+    shape,
+    strides,
+    suboffsets,
+    converter,
+  });
+  return Module.hiwire.new_value(result);
+});
+// clang-format on
 
-static JsRef
-_convert_float32(char* data)
+JsRef
+_python2js_buffer(PyObject* x)
 {
-  float v = *((float*)data);
-  return hiwire_double(v);
+  Py_buffer view;
+  if (PyObject_GetBuffer(x, &view, PyBUF_FULL_RO) == -1) {
+    return NULL;
+  }
+  // clang-format off
+  JsRef result = _python2js_buffer_inner(
+    view.buf,
+    view.itemsize,
+    view.ndim,
+    view.format,
+    view.shape,
+    view.strides,
+    view.suboffsets
+  );
+  // clang-format on
+  PyBuffer_Release(&view);
+  return result;
 }
 
-static JsRef
-_convert_float32_swap(char* data)
-{
-  union float32_t
+EM_JS_NUM(errcode, python2js_buffer_init, (), {
+  Module.python2js_buffer_1d_contiguous = function(ptr, stride, n, converter)
   {
-    u32 i;
-    float f;
-  } v;
-
-  v.f = *((float*)data);
-  v.i = be32toh(v.i);
-  return hiwire_double(v.f);
-}
-
-static JsRef
-_convert_float64(char* data)
-{
-  double v = *((double*)data);
-  return hiwire_double(v);
-}
-
-static JsRef
-_convert_float64_swap(char* data)
-{
-  union float64_t
+    "use strict";
+    let byteLength = stride * n;
+    let backing = HEAP8.slice(ptr, ptr + byteLength).buffer;
+    return converter(backing);
+  };
+
+  Module.python2js_buffer_1d_noncontiguous =
+    function(ptr, stride, suboffset, n, itemsize, converter)
   {
-    u64 i;
-    double f;
-  } v;
-
-  v.f = *((double*)data);
-  v.i = be64toh(v.i);
-  return hiwire_double(v.f);
-}
-
-static scalar_converter*
-_python2js_buffer_get_converter(Py_buffer* buff)
-{
-  // Uses Python's struct typecodes as defined here:
-  // https://docs.python.org/3.8/library/array.html
-
-  char format;
-  char swap;
-  if (buff->format == NULL) {
-    swap = 0;
-    format = 'B';
-  } else {
-    switch (buff->format[0]) {
-      case '>':
-      case '!':
-        swap = 1;
-        format = buff->format[1];
-        break;
-      case '=':
-      case '<':
-      case '@':
-        swap = 0;
-        format = buff->format[1];
-        break;
-      default:
-        swap = 0;
-        format = buff->format[0];
-    }
-  }
-
-  switch (format) {
-    case 'c':
-    case 'b':
-      return _convert_int8;
-    case 'B':
-      return _convert_uint8;
-    case '?':
-      return _convert_bool;
-    case 'h':
-      if (swap) {
-        return _convert_int16_swap;
-      } else {
-        return _convert_int16;
+    "use strict";
+    let byteLength = itemsize * n;
+    let buffer = new Uint8Array(byteLength);
+    for (i = 0; i < n; ++i) {
+      let curptr = ptr + i * stride;
+      if (suboffset >= 0) {
+        curptr = HEAP32[curptr / 4] + suboffset;
       }
-    case 'H':
-      if (swap) {
-        return _convert_uint16_swap;
-      } else {
-        return _convert_uint16;
-      }
-    case 'i':
-    case 'l':
-    case 'n':
-      if (swap) {
-        return _convert_int32_swap;
-      } else {
-        return _convert_int32;
-      }
-    case 'I':
-    case 'L':
-    case 'N':
-      if (swap) {
-        return _convert_uint32_swap;
-      } else {
-        return _convert_uint32;
-      }
-    case 'q':
-      if (swap) {
-        return _convert_int64_swap;
-      } else {
-        return _convert_int64;
-      }
-    case 'Q':
-      if (swap) {
-        return _convert_uint64_swap;
-      } else {
-        return _convert_uint64;
-      }
-    case 'f':
-      if (swap) {
-        return _convert_float32_swap;
-      } else {
-        return _convert_float32;
-      }
-    case 'd':
-      if (swap) {
-        return _convert_float64_swap;
-      } else {
-        return _convert_float64;
-      }
-    default:
-      return NULL;
-  }
-}
-
-static JsRef
-_python2js_buffer_recursive(Py_buffer* buff,
-                            char* ptr,
-                            int dim,
-                            scalar_converter* convert)
-{
-  // This function is basically a manual conversion of `recursive_tolist` in
-  // Numpy to use the Python buffer interface and output Javascript.
-
-  Py_ssize_t i, n, stride;
-  JsRef jsarray, jsitem;
-
-  if (dim >= buff->ndim) {
-    return convert(ptr);
-  }
-
-  n = buff->shape[dim];
-  stride = buff->strides[dim];
-
-  jsarray = hiwire_array();
-
-  for (i = 0; i < n; ++i) {
-    jsitem = _python2js_buffer_recursive(buff, ptr, dim + 1, convert);
-    if (jsitem == NULL) {
-      hiwire_decref(jsarray);
-      return NULL;
+      buffer.set(HEAP8.subarray(curptr, curptr + itemsize), i * itemsize);
     }
-    hiwire_push_array(jsarray, jsitem);
-    hiwire_decref(jsitem);
+    return converter(buffer.buffer);
+  };
 
-    ptr += stride;
-  }
-
-  return jsarray;
-}
-
-static JsRef
-_python2js_buffer_to_typed_array(Py_buffer* buff)
-{
-  // Uses Python's struct typecodes as defined here:
-  // https://docs.python.org/3.8/library/array.html
-
-  char format;
-  if (buff->format == NULL) {
-    format = 'B';
-  } else {
-    switch (buff->format[0]) {
-      case '>':
-      case '!':
-        // This path can't handle byte-swapping
-        return NULL;
-      case '=':
-      case '<':
-      case '@':
-        format = buff->format[1];
-        break;
-      default:
-        format = buff->format[0];
+  Module._python2js_buffer_recursive = function(ptr, curdim, bufferData)
+  {
+    "use strict";
+    let n = HEAP32[bufferData.shape / 4 + curdim];
+    let stride = HEAP32[bufferData.strides / 4 + curdim];
+    let suboffset = -1;
+    // clang-format off
+    if (bufferData.suboffsets !== 0) {
+      suboffset = HEAP32[bufferData.suboffsets / 4 + curdim];
     }
-  }
-
-  Py_ssize_t len = buff->len / buff->itemsize;
-  switch (format) {
-    case 'c':
-    case 'b':
-      return hiwire_int8array((i8*)buff->buf, len);
-    case 'B':
-      return hiwire_uint8array((u8*)buff->buf, len);
-    case '?':
-      return NULL;
-    case 'h':
-      return hiwire_int16array((i16*)buff->buf, len);
-    case 'H':
-      return hiwire_uint16array((u16*)buff->buf, len);
-    case 'i':
-    case 'l':
-    case 'n':
-      return hiwire_int32array((i32*)buff->buf, len);
-    case 'I':
-    case 'L':
-    case 'N':
-      return hiwire_uint32array((u32*)buff->buf, len);
-    case 'q':
-    case 'Q':
-      return NULL;
-    case 'f':
-      return hiwire_float32array((f32*)buff->buf, len);
-    case 'd':
-      return hiwire_float64array((f64*)buff->buf, len);
-    default:
-      return NULL;
-  }
-}
-
-enum shareable_enum
-{
-  NOT_SHAREABLE,
-  CONTIGUOUS,
-  NOT_CONTIGUOUS
-};
-
-static JsRef
-_python2js_shareable_buffer_recursive(Py_buffer* buff,
-                                      enum shareable_enum shareable,
-                                      JsRef idarr,
-                                      int ptr,
-                                      int dim)
-{
-  Py_ssize_t i, n, stride;
-  JsRef jsarray, jsitem;
-
-  switch (shareable) {
-    case NOT_CONTIGUOUS:
-      if (dim >= buff->ndim) {
-        // The last dimension isn't contiguous, so we need to output one-by-one
-        return hiwire_get_member_int(idarr, ptr / buff->itemsize);
-      }
-      break;
-    case CONTIGUOUS:
-      if (dim == buff->ndim - 1) {
-        // The last dimension is contiguous, so we can output a whole row at a
-        // time
-        return hiwire_subarray(
-          idarr, ptr / buff->itemsize, ptr / buff->itemsize + buff->shape[dim]);
+    if (curdim === bufferData.ndim - 1) {
+      if (stride === bufferData.itemsize && suboffset < 0) {
+        // clang-format on
+        return Module.python2js_buffer_1d_contiguous(
+          ptr, stride, n, bufferData.converter);
+      } else {
+        return Module.python2js_buffer_1d_noncontiguous(
+          ptr, stride, suboffset, n, bufferData.itemsize, bufferData.converter);
       }
-      break;
-    default:
-      break;
-  }
-
-  n = buff->shape[dim];
-  stride = buff->strides[dim];
-
-  jsarray = hiwire_array();
-
-  for (i = 0; i < n; ++i) {
-    jsitem = _python2js_shareable_buffer_recursive(
-      buff, shareable, idarr, ptr, dim + 1);
-    if (jsitem == NULL) {
-      hiwire_decref(jsarray);
-      return NULL;
     }
-    hiwire_push_array(jsarray, jsitem);
-    hiwire_decref(jsitem);
-
-    ptr += stride;
-  }
-
-  return jsarray;
-}
 
-static enum shareable_enum
-_python2js_buffer_is_shareable(Py_buffer* buff)
-{
-  if (buff->ndim == 0) {
-    return NOT_SHAREABLE;
-  }
-
-  char* invalid_codes = ">!qQ?";
-  for (char* i = buff->format; *i != 0; ++i) {
-    for (char* j = invalid_codes; *j != 0; ++j) {
-      if (*i == *j) {
-        return NOT_SHAREABLE;
+    let result = [];
+    for (let i = 0; i < n; ++i) {
+      let curPtr = ptr + i * stride;
+      if (suboffset >= 0) {
+        curptr = HEAP32[curptr / 4] + suboffset;
       }
+      result.push(
+        Module._python2js_buffer_recursive(curPtr, curdim + 1, bufferData));
     }
-  }
+    return result;
+  };
 
-  for (int i = 0; i < buff->ndim; ++i) {
-    if (buff->strides[i] <= 0) {
-      return NOT_SHAREABLE;
+  Module.get_converter = function(format, itemsize)
+  {
+    "use strict";
+    let formatStr = UTF8ToString(format);
+    if (formatStr.length > 2) {
+      throw new Error(
+        `Expected format string to have length <= 2, got '${formatStr}'`);
     }
-  }
-
-  if (buff->itemsize != buff->strides[buff->ndim - 1]) {
-    return NOT_CONTIGUOUS;
-  }
-
-  // We can use the most efficient method
-  return CONTIGUOUS;
-}
-
-JsRef
-_python2js_buffer(PyObject* x)
-{
-  PyObject* memoryview = PyMemoryView_FromObject(x);
-  if (memoryview == NULL) {
-    PyErr_Clear();
-    return NULL;
-  }
-
-  Py_buffer* buff;
-  buff = PyMemoryView_GET_BUFFER(memoryview);
-
-  enum shareable_enum shareable = _python2js_buffer_is_shareable(buff);
-  JsRef result;
-
-  if (shareable != NOT_SHAREABLE) {
-    JsRef idarr = _python2js_buffer_to_typed_array(buff);
-    if (idarr == NULL) {
-      PyErr_SetString(
-        PyExc_TypeError,
-        "Internal error: Invalid type to convert to array buffer.");
-      return NULL;
+    let formatChar = formatStr.slice(-1);
+    let alignChar = formatStr.slice(0, -1);
+    // clang-format off
+    let swap = alignChar === "!" || alignChar === ">";
+    // clang-format on
+    let swapFunc;
+    if (!swap) {
+      // clang-format off
+      swapFunc = buff => buff;
+      // clang-format on
+    } else {
+      let getFuncName;
+      let setFuncName;
+      switch (itemsize) {
+        case 2:
+          getFuncName = "getUint16";
+          setFuncName = "setUint16";
+          break;
+        case 4:
+          getFuncName = "getUint32";
+          setFuncName = "setUint32";
+          break;
+        case 8:
+          getFuncName = "getFloat64";
+          setFuncName = "setFloat64";
+          break;
+      }
+      swapFunc = function(buff, size)
+      {
+        let dataview = new DataView(buff);
+        let getFunc = dataview[getFuncName].bind(dataview);
+        let setFunc = dataview[setFuncName].bind(dataview);
+        for (let byte = 0; byte < dataview.byteLength; byte += itemsize) {
+          // Get value as little endian, set back as big endian.
+          setFunc(byte, getFunc(byte, true), false);
+        }
+        return buff;
+      }
     }
-
-    result =
-      _python2js_shareable_buffer_recursive(buff, shareable, idarr, 0, 0);
-    hiwire_decref(idarr);
-  } else {
-    scalar_converter* convert = _python2js_buffer_get_converter(buff);
-    if (convert == NULL) {
-      Py_DECREF(memoryview);
-      return NULL;
+    // clang-format off
+    switch (formatChar) {
+      case 'c':
+        let decoder = new TextDecoder("utf8");
+        return (buff) => decoder.decode(buff);
+      case 'b':
+        return (buff) => new Int8Array(buff);
+      case 'B':
+        return (buff) => new Uint8Array(buff);
+      case '?':
+        return (buff) => Array.from(new Uint8Array(buff)).map(x => !!x);
+      case 'h':
+        return (buff) => new Int16Array(swapFunc(buff));
+      case 'H':
+        return (buff) => new Uint16Array(swapFunc(buff));
+      case 'i':
+      case 'l':
+      case 'n':
+        return (buff) => new Int32Array(swapFunc(buff));
+        return "getInt32";
+      case 'I':
+      case 'L':
+      case 'N':
+        return (buff) => new Uint32Array(swapFunc(buff));
+      case 'q':
+        return (buff) => new BigInt64Array(swapFunc(buff));
+      case 'Q':
+        return (buff) => new BigUint64Array(swapFunc(buff));
+      case 'f':
+        return (buff) => new Float32Array(swapFunc(buff));
+      case 'd':
+        return (buff) => new Float64Array(swapFunc(buff));
+      default:
+        throw new Error(`Unrecognized format character '${formatChar}'`);
     }
-
-    result = _python2js_buffer_recursive(buff, buff->buf, 0, convert);
-  }
-
-  Py_DECREF(memoryview);
-
-  return result;
-}
+    // clang-format on
+  };
+});
diff --git a/src/core/python2js_buffer.h b/src/core/python2js_buffer.h
index cd1fc138612..2c51cb9f507 100644
--- a/src/core/python2js_buffer.h
+++ b/src/core/python2js_buffer.h
@@ -18,4 +18,7 @@
 JsRef
 _python2js_buffer(PyObject* x);
 
+errcode
+python2js_buffer_init();
+
 #endif /* PYTHON2JS_BUFFER_H */


Code review comments:
diff_hunk: @@ -272,6 +272,7 @@ performs the following explicit conversions:
 | `list`, `tuple`  | `Array`             |
 | `dict`           | `Map`               |
 | `set`            | `Set`               |
+| a buffer         | `TypedArray`        |
path: docs/usage/type-conversions.md
text: @user1:
Maybe give An object with buffer interface such as bytes, numpy.array

diff_hunk: @@ -12,487 +13,195 @@
 // This file handles the conversion of Python buffer objects (which loosely
 // represent Numpy arrays) to Javascript.
 
-// There are two methods here:
-
-//   1. Converts everything to nested Javascript arrays, where the scalars are
+//   Converts everything to nested Javascript arrays, where the scalars are
 //   standard Javascript numbers (python2js_buffer_recursive)
 
-//   2. Converts everything to nested arrays, where the last contiguous
-//   dimension is a subarray of a TypedArray that points to the original bytes
-//   on the WebAssembly (Python) side. This is much faster since it doesn't
-//   require copying the data, and the data is shared. In the case of a
-//   one-dimensional array, the result is simply a TypedArray. Unfortunately,
-//   this requires that the source array is C-contiguous and in native (little)
-//   endian order. (python2js_shareable_buffer_recursive)
-
-// Unfortunately, this also means that there are different semantics: sometimes
-// the array is a copy, and other times it is a shared reference. One should
-// write code that doesn't rely on either behavior, but treats this simply as
-// the performance optimization that it is.
-
-typedef JsRef(scalar_converter)(char*);
-
-static JsRef
-_convert_bool(char* data)
-{
-  char v = *((char*)data);
-  return hiwire_bool((int)v);
-}
-
-static JsRef
-_convert_int8(char* data)
-{
-  i8 v = *((i8*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint8(char* data)
-{
-  u8 v = *((u8*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int16(char* data)
-{
-  i16 v = *((i16*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int16_swap(char* data)
-{
-  i16 v = *((i16*)data);
-  return hiwire_int(be16toh(v));
-}
-
-static JsRef
-_convert_uint16(char* data)
-{
-  u16 v = *((u16*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint16_swap(char* data)
-{
-  u16 v = *((u16*)data);
-  return hiwire_int(be16toh(v));
-}
-
-static JsRef
-_convert_int32(char* data)
-{
-  i32 v = *((i32*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int32_swap(char* data)
-{
-  i32 v = *((i32*)data);
-  return hiwire_int(be32toh(v));
-}
-
-static JsRef
-_convert_uint32(char* data)
-{
-  u32 v = *((u32*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint32_swap(char* data)
-{
-  u32 v = *((u32*)data);
-  return hiwire_int(be32toh(v));
-}
-
-static JsRef
-_convert_int64(char* data)
-{
-  i64 v = *((i64*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_int64_swap(char* data)
-{
-  i64 v = *((i64*)data);
-  return hiwire_int(be64toh(v));
-}
-
-static JsRef
-_convert_uint64(char* data)
-{
-  u64 v = *((u64*)data);
-  return hiwire_int(v);
-}
-
-static JsRef
-_convert_uint64_swap(char* data)
-{
-  u64 v = *((u64*)data);
-  return hiwire_int(be64toh(v));
-}
+// clang-format off
+EM_JS_REF(JsRef, _python2js_buffer_inner, (
+  void* buf,
+  Py_ssize_t itemsize,
+  int ndim,
+  char* format,
+  Py_ssize_t* shape,
+  Py_ssize_t* strides,
+  Py_ssize_t* suboffsets
+), {
+  let converter = Module.get_converter(format, itemsize);
+  let result = Module._python2js_buffer_recursive(buf, 0, {
+    ndim,
+    format,
+    itemsize,
+    shape,
+    strides,
+    suboffsets,
+    converter,
+  });
+  return Module.hiwire.new_value(result);
+});
+// clang-format on
 
-static JsRef
-_convert_float32(char* data)
+JsRef
+_python2js_buffer(PyObject* x)
 {
-  float v = *((float*)data);
-  return hiwire_double(v);
+  Py_buffer view;
+  if (PyObject_GetBuffer(x, &view, PyBUF_FULL_RO) == -1) {
+    return NULL;
+  }
+  // clang-format off
+  JsRef result = _python2js_buffer_inner(
+    view.buf,
+    view.itemsize,
+    view.ndim,
+    view.format,
+    view.shape,
+    view.strides,
+    view.suboffsets
+  );
+  // clang-format on
+  PyBuffer_Release(&view);
+  return result;
 }
 
-static JsRef
-_convert_float32_swap(char* data)
-{
-  union float32_t
+EM_JS_NUM(errcode, python2js_buffer_init, (), {
+  Module.python2js_buffer_1d_contiguous = function(ptr, stride, n, converter)
   {
-    u32 i;
-    float f;
-  } v;
-
-  v.f = *((float*)data);
-  v.i = be32toh(v.i);
-  return hiwire_double(v.f);
-}
-
-static JsRef
-_convert_float64(char* data)
-{
-  double v = *((double*)data);
-  return hiwire_double(v);
-}
-
-static JsRef
-_convert_float64_swap(char* data)
-{
-  union float64_t
+    "use strict";
+    let byteLength = stride * n;
+    let backing = HEAP8.slice(ptr, ptr + byteLength).buffer;
+    return converter(backing);
+  };
+
+  Module.python2js_buffer_1d_noncontiguous =
+    function(ptr, stride, suboffset, n, itemsize, converter)
   {
-    u64 i;
-    double f;
-  } v;
-
-  v.f = *((double*)data);
-  v.i = be64toh(v.i);
-  return hiwire_double(v.f);
-}
-
-static scalar_converter*
-_python2js_buffer_get_converter(Py_buffer* buff)
-{
-  // Uses Python's struct typecodes as defined here:
-  // https://docs.python.org/3.8/library/array.html
-
-  char format;
-  char swap;
-  if (buff->format == NULL) {
-    swap = 0;
-    format = 'B';
-  } else {
-    switch (buff->format[0]) {
-      case '>':
-      case '!':
-        swap = 1;
-        format = buff->format[1];
-        break;
-      case '=':
-      case '<':
-      case '@':
-        swap = 0;
-        format = buff->format[1];
-        break;
-      default:
-        swap = 0;
-        format = buff->format[0];
-    }
-  }
-
-  switch (format) {
-    case 'c':
-    case 'b':
-      return _convert_int8;
-    case 'B':
-      return _convert_uint8;
-    case '?':
-      return _convert_bool;
-    case 'h':
-      if (swap) {
-        return _convert_int16_swap;
-      } else {
-        return _convert_int16;
+    "use strict";
+    let byteLength = itemsize * n;
+    let buffer = new Uint8Array(byteLength);
+    for (i = 0; i < n; ++i) {
+      let curptr = ptr + i * stride;
+      if (suboffset >= 0) {
+        curptr = HEAP32[curptr / 4] + suboffset;
       }
-    case 'H':
-      if (swap) {
-        return _convert_uint16_swap;
-      } else {
-        return _convert_uint16;
-      }
-    case 'i':
-    case 'l':
-    case 'n':
-      if (swap) {
-        return _convert_int32_swap;
-      } else {
-        return _convert_int32;
-      }
-    case 'I':
-    case 'L':
-    case 'N':
-      if (swap) {
-        return _convert_uint32_swap;
-      } else {
-        return _convert_uint32;
-      }
-    case 'q':
-      if (swap) {
-        return _convert_int64_swap;
-      } else {
-        return _convert_int64;
-      }
-    case 'Q':
-      if (swap) {
-        return _convert_uint64_swap;
-      } else {
-        return _convert_uint64;
-      }
-    case 'f':
-      if (swap) {
-        return _convert_float32_swap;
-      } else {
-        return _convert_float32;
-      }
-    case 'd':
-      if (swap) {
-        return _convert_float64_swap;
-      } else {
-        return _convert_float64;
-      }
-    default:
-      return NULL;
-  }
-}
-
-static JsRef
-_python2js_buffer_recursive(Py_buffer* buff,
-                            char* ptr,
-                            int dim,
-                            scalar_converter* convert)
-{
-  // This function is basically a manual conversion of `recursive_tolist` in
-  // Numpy to use the Python buffer interface and output Javascript.
-
-  Py_ssize_t i, n, stride;
-  JsRef jsarray, jsitem;
-
-  if (dim >= buff->ndim) {
-    return convert(ptr);
-  }
-
-  n = buff->shape[dim];
-  stride = buff->strides[dim];
-
-  jsarray = hiwire_array();
-
-  for (i = 0; i < n; ++i) {
-    jsitem = _python2js_buffer_recursive(buff, ptr, dim + 1, convert);
-    if (jsitem == NULL) {
-      hiwire_decref(jsarray);
-      return NULL;
+      buffer.set(HEAP8.subarray(curptr, curptr + itemsize), i * itemsize);
     }
-    hiwire_push_array(jsarray, jsitem);
-    hiwire_decref(jsitem);
+    return converter(buffer.buffer);
+  };
 
-    ptr += stride;
-  }
-
-  return jsarray;
-}
-
-static JsRef
-_python2js_buffer_to_typed_array(Py_buffer* buff)
-{
-  // Uses Python's struct typecodes as defined here:
-  // https://docs.python.org/3.8/library/array.html
-
-  char format;
-  if (buff->format == NULL) {
-    format = 'B';
-  } else {
-    switch (buff->format[0]) {
-      case '>':
-      case '!':
-        // This path can't handle byte-swapping
-        return NULL;
-      case '=':
-      case '<':
-      case '@':
-        format = buff->format[1];
-        break;
-      default:
-        format = buff->format[0];
+  Module._python2js_buffer_recursive = function(ptr, curdim, bufferData)
+  {
+    "use strict";
+    let n = HEAP32[bufferData.shape / 4 + curdim];
+    let stride = HEAP32[bufferData.strides / 4 + curdim];
+    let suboffset = -1;
+    // clang-format off
+    if (bufferData.suboffsets !== 0) {
+      suboffset = HEAP32[bufferData.suboffsets / 4 + curdim];
     }
-  }
-
-  Py_ssize_t len = buff->len / buff->itemsize;
-  switch (format) {
-    case 'c':
-    case 'b':
-      return hiwire_int8array((i8*)buff->buf, len);
-    case 'B':
-      return hiwire_uint8array((u8*)buff->buf, len);
-    case '?':
-      return NULL;
-    case 'h':
-      return hiwire_int16array((i16*)buff->buf, len);
-    case 'H':
-      return hiwire_uint16array((u16*)buff->buf, len);
-    case 'i':
-    case 'l':
-    case 'n':
-      return hiwire_int32array((i32*)buff->buf, len);
-    case 'I':
-    case 'L':
-    case 'N':
-      return hiwire_uint32array((u32*)buff->buf, len);
-    case 'q':
-    case 'Q':
-      return NULL;
-    case 'f':
-      return hiwire_float32array((f32*)buff->buf, len);
-    case 'd':
-      return hiwire_float64array((f64*)buff->buf, len);
-    default:
-      return NULL;
-  }
-}
-
-enum shareable_enum
-{
-  NOT_SHAREABLE,
-  CONTIGUOUS,
-  NOT_CONTIGUOUS
-};
-
-static JsRef
-_python2js_shareable_buffer_recursive(Py_buffer* buff,
-                                      enum shareable_enum shareable,
-                                      JsRef idarr,
-                                      int ptr,
-                                      int dim)
-{
-  Py_ssize_t i, n, stride;
-  JsRef jsarray, jsitem;
-
-  switch (shareable) {
-    case NOT_CONTIGUOUS:
-      if (dim >= buff->ndim) {
-        // The last dimension isn't contiguous, so we need to output one-by-one
-        return hiwire_get_member_int(idarr, ptr / buff->itemsize);
-      }
-      break;
-    case CONTIGUOUS:
-      if (dim == buff->ndim - 1) {
-        // The last dimension is contiguous, so we can output a whole row at a
-        // time
-        return hiwire_subarray(
-          idarr, ptr / buff->itemsize, ptr / buff->itemsize + buff->shape[dim]);
+    if (curdim === bufferData.ndim - 1) {
+      if (stride === bufferData.itemsize && suboffset < 0) {
+        // clang-format on
+        return Module.python2js_buffer_1d_contiguous(
+          ptr, stride, n, bufferData.converter);
+      } else {
+        return Module.python2js_buffer_1d_noncontiguous(
+          ptr, stride, suboffset, n, bufferData.itemsize, bufferData.converter);
       }
-      break;
-    default:
-      break;
-  }
-
-  n = buff->shape[dim];
-  stride = buff->strides[dim];
-
-  jsarray = hiwire_array();
-
-  for (i = 0; i < n; ++i) {
-    jsitem = _python2js_shareable_buffer_recursive(
-      buff, shareable, idarr, ptr, dim + 1);
-    if (jsitem == NULL) {
-      hiwire_decref(jsarray);
-      return NULL;
     }
-    hiwire_push_array(jsarray, jsitem);
-    hiwire_decref(jsitem);
-
-    ptr += stride;
-  }
-
-  return jsarray;
-}
 
-static enum shareable_enum
-_python2js_buffer_is_shareable(Py_buffer* buff)
-{
-  if (buff->ndim == 0) {
-    return NOT_SHAREABLE;
-  }
-
-  char* invalid_codes = ">!qQ?";
-  for (char* i = buff->format; *i != 0; ++i) {
-    for (char* j = invalid_codes; *j != 0; ++j) {
-      if (*i == *j) {
-        return NOT_SHAREABLE;
+    let result = [];
+    for (let i = 0; i < n; ++i) {
+      let curPtr = ptr + i * stride;
+      if (suboffset >= 0) {
+        curptr = HEAP32[curptr / 4] + suboffset;
       }
+      result.push(
+        Module._python2js_buffer_recursive(curPtr, curdim + 1, bufferData));
     }
-  }
+    return result;
+  };
 
-  for (int i = 0; i < buff->ndim; ++i) {
-    if (buff->strides[i] <= 0) {
-      return NOT_SHAREABLE;
+  Module.get_converter = function(format, itemsize)
+  {
+    "use strict";
+    let formatStr = UTF8ToString(format);
+    if (formatStr.length > 2) {
+      throw new Error(
+        `Expected format string to have length <= 2, got '${formatStr}'`);
     }
-  }
-
-  if (buff->itemsize != buff->strides[buff->ndim - 1]) {
-    return NOT_CONTIGUOUS;
-  }
-
-  // We can use the most efficient method
-  return CONTIGUOUS;
-}
-
-JsRef
-_python2js_buffer(PyObject* x)
-{
-  PyObject* memoryview = PyMemoryView_FromObject(x);
-  if (memoryview == NULL) {
-    PyErr_Clear();
-    return NULL;
-  }
-
-  Py_buffer* buff;
-  buff = PyMemoryView_GET_BUFFER(memoryview);
-
-  enum shareable_enum shareable = _python2js_buffer_is_shareable(buff);
-  JsRef result;
-
-  if (shareable != NOT_SHAREABLE) {
-    JsRef idarr = _python2js_buffer_to_typed_array(buff);
-    if (idarr == NULL) {
-      PyErr_SetString(
-        PyExc_TypeError,
-        "Internal error: Invalid type to convert to array buffer.");
-      return NULL;
+    let formatChar = formatStr.slice(-1);
+    let alignChar = formatStr.slice(0, -1);
+    // clang-format off
+    let swap = alignChar === "!" || alignChar === ">";
+    // clang-format on
+    let swapFunc;
+    if (!swap) {
+      // clang-format off
+      swapFunc = buff => buff;
+      // clang-format on
+    } else {
+      let getFuncName;
+      let setFuncName;
+      switch (itemsize) {
+        case 2:
+          getFuncName = "getUint16";
+          setFuncName = "setUint16";
+          break;
+        case 4:
+          getFuncName = "getUint32";
+          setFuncName = "setUint32";
+          break;
+        case 8:
+          getFuncName = "getFloat64";
+          setFuncName = "setFloat64";
+          break;
+      }
+      swapFunc = function(buff, size)
+      {
+        let dataview = new DataView(buff);
+        let getFunc = dataview[getFuncName].bind(dataview);
+        let setFunc = dataview[setFuncName].bind(dataview);
+        for (let byte = 0; byte < dataview.byteLength; byte += itemsize) {
+          // Get value as little endian, set back as big endian.
+          setFunc(byte, getFunc(byte, true), false);
+        }
+        return buff;
+      }
     }
-
-    result =
-      _python2js_shareable_buffer_recursive(buff, shareable, idarr, 0, 0);
-    hiwire_decref(idarr);
-  } else {
-    scalar_converter* convert = _python2js_buffer_get_converter(buff);
-    if (convert == NULL) {
-      Py_DECREF(memoryview);
-      return NULL;
+    // clang-format off
+    switch (formatChar) {
+      case 'c':
+        let decoder = new TextDecoder("utf8");
+        return (buff) => decoder.decode(buff);
+      case 'b':
+        return (buff) => new Int8Array(buff);
+      case 'B':
+        return (buff) => new Uint8Array(buff);
+      case '?':
+        return (buff) => Array.from(new Uint8Array(buff)).map(x => !!x);
+      case 'h':
+        return (buff) => new Int16Array(swapFunc(buff));
+      case 'H':
+        return (buff) => new Uint16Array(swapFunc(buff));
+      case 'i':
+      case 'l':
+      case 'n':
+        return (buff) => new Int32Array(swapFunc(buff));
+        return "getInt32";
+      case 'I':
+      case 'L':
+      case 'N':
+        return (buff) => new Uint32Array(swapFunc(buff));
+      case 'q':
+        return (buff) => new BigInt64Array(swapFunc(buff));
+      case 'Q':
+        return (buff) => new BigUint64Array(swapFunc(buff));
path: src/core/python2js_buffer.c
text: @user1:
It should probably check if bigint is available and error otherwise)

@author:
Well it will already throw a `ReferenceError` if `BigInt64Array` is undefined =P