Repository: python/cpython
Pull request #133237: gh-61103: don't use native complex types in ctypes
Author: login: skirpichev
Labels: name: skip news
Created: 2025-05-01T04:29:04Z | Merged: 2025-05-05T09:23:30Z

* drop _complex.h header
* use appropriate real arrays to replace complex types

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

# Pull Request title

It should be in the following format:

```
gh-NNNNNN: Summary of the changes made
```

Where: gh-NNNNNN refers to the GitHub issue number.

Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.

# Backport Pull Request title

If this is a backport PR (PR made against branches other than `main`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNNNN)
```

Where: [X.Y] is the branch name, for example: [3.13].

GH-NNNNNN refers to the PR number from `main`.

-->


<!-- gh-issue-number: gh-61103 -->
* Issue: gh-61103
<!-- /gh-issue-number -->


<!-- readthedocs-preview cpython-previews start -->
----
📚 Documentation preview 📚: https://cpython-previews--133237.org.readthedocs.build/

<!-- readthedocs-preview cpython-previews end -->

Note, that `Py_FFI_SUPPORT_C_COMPLEX` check configure check imply support for complex types.  So, `Py_HAVE_C_COMPLEX` check is redundant.  Though, I'm not sure if it worth removing.


Diff:
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 37ce0b55203cf7..0484605731cc4b 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -3313,7 +3313,7 @@ MODULE_CMATH_DEPS=$(srcdir)/Modules/_math.h
 MODULE_MATH_DEPS=$(srcdir)/Modules/_math.h
 MODULE_PYEXPAT_DEPS=@LIBEXPAT_INTERNAL@
 MODULE_UNICODEDATA_DEPS=$(srcdir)/Modules/unicodedata_db.h $(srcdir)/Modules/unicodename_db.h
-MODULE__CTYPES_DEPS=$(srcdir)/Modules/_ctypes/ctypes.h $(srcdir)/Modules/_complex.h
+MODULE__CTYPES_DEPS=$(srcdir)/Modules/_ctypes/ctypes.h
 MODULE__CTYPES_TEST_DEPS=$(srcdir)/Modules/_ctypes/_ctypes_test_generated.c.h
 MODULE__CTYPES_MALLOC_CLOSURE=@MODULE__CTYPES_MALLOC_CLOSURE@
 MODULE__DECIMAL_DEPS=$(srcdir)/Modules/_decimal/docstrings.h @LIBMPDEC_INTERNAL@
diff --git a/Modules/_complex.h b/Modules/_complex.h
deleted file mode 100644
index 28d4a32794b97c..00000000000000
--- a/Modules/_complex.h
+++ /dev/null
@@ -1,54 +0,0 @@
-/* Workarounds for buggy complex number arithmetic implementations. */
-
-#ifndef Py_HAVE_C_COMPLEX
-#  error "this header file should only be included if Py_HAVE_C_COMPLEX is defined"
-#endif
-
-#include <complex.h>
-
-/* Other compilers (than clang), that claims to
-   implement C11 *and* define __STDC_IEC_559_COMPLEX__ - don't have
-   issue with CMPLX().  This is specific to glibc & clang combination:
-   https://sourceware.org/bugzilla/show_bug.cgi?id=26287
-
-   Here we fallback to using __builtin_complex(), available in clang
-   v12+.  Else CMPLX implemented following C11 6.2.5p13: "Each complex type
-   has the same representation and alignment requirements as an array
-   type containing exactly two elements of the corresponding real type;
-   the first element is equal to the real part, and the second element
-   to the imaginary part, of the complex number.
- */
-#if !defined(CMPLX)
-#  if defined(__clang__) && __has_builtin(__builtin_complex)
-#    define CMPLX(x, y) __builtin_complex ((double) (x), (double) (y))
-#    define CMPLXF(x, y) __builtin_complex ((float) (x), (float) (y))
-#    define CMPLXL(x, y) __builtin_complex ((long double) (x), (long double) (y))
-#  else
-static inline double complex
-CMPLX(double real, double imag)
-{
-    double complex z;
-    ((double *)(&z))[0] = real;
-    ((double *)(&z))[1] = imag;
-    return z;
-}
-
-static inline float complex
-CMPLXF(float real, float imag)
-{
-    float complex z;
-    ((float *)(&z))[0] = real;
-    ((float *)(&z))[1] = imag;
-    return z;
-}
-
-static inline long double complex
-CMPLXL(long double real, long double imag)
-{
-    long double complex z;
-    ((long double *)(&z))[0] = real;
-    ((long double *)(&z))[1] = imag;
-    return z;
-}
-#  endif
-#endif
diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c
index 2268072545f45d..557bfa85b1ea29 100644
--- a/Modules/_ctypes/_ctypes_test.c
+++ b/Modules/_ctypes/_ctypes_test.c
@@ -24,7 +24,7 @@
 #endif
 
 #if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-#  include "../_complex.h"        // csqrt()
+#  include <complex.h>            // csqrt()
 #  undef I                        // for _ctypes_test_generated.c.h
 #endif
 #include <stdio.h>                // printf()
diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
index 83471aa3a42ad2..619e4ad08a65df 100644
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -103,9 +103,6 @@ module _ctypes
 #include "pycore_global_objects.h"// _Py_ID()
 #include "pycore_traceback.h"     // _PyTraceback_Add()
 
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-#include "../_complex.h"          // complex
-#endif
 #define clinic_state() (get_module_state(module))
 #include "clinic/callproc.c.h"
 #undef clinic_state
@@ -652,11 +649,9 @@ union result {
     double d;
     float f;
     void *p;
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-    double complex D;
-    float complex F;
-    long double complex G;
-#endif
+    double D[2];
+    float F[2];
+    long double G[2];
 };
 
 struct argument {
diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c
index 580ea18af2a43a..86bcc805360a3f 100644
--- a/Modules/_ctypes/cfield.c
+++ b/Modules/_ctypes/cfield.c
@@ -14,10 +14,6 @@
 #include <ffi.h>
 #include "ctypes.h"
 
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-#  include "../_complex.h"        // complex
-#endif
-
 #define CTYPES_CFIELD_CAPSULE_NAME_PYMEM "_ctypes/cfield.c pymem"
 
 /*[clinic input]
@@ -763,18 +759,25 @@ d_get(void *ptr, Py_ssize_t size)
     return PyFloat_FromDouble(val);
 }
 
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
+#if defined(Py_FFI_SUPPORT_C_COMPLEX)
+
+/* We don't use _Complex types here, using arrays instead, as the C11+
+   standard says: "Each complex type has the same representation and alignment
+   requirements as an array type containing exactly two elements of the
+   corresponding real type; the first element is equal to the real part, and
+   the second element to the imaginary part, of the complex number." */
+
 /* D: double complex */
 static PyObject *
 D_set(void *ptr, PyObject *value, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(double complex)));
+    assert(NUM_BITS(size) || (size == 2*sizeof(double)));
     Py_complex c = PyComplex_AsCComplex(value);
 
     if (c.real == -1 && PyErr_Occurred()) {
         return NULL;
     }
-    double complex x = CMPLX(c.real, c.imag);
+    double x[2] = {c.real, c.imag};
     memcpy(ptr, &x, sizeof(x));
     _RET(value);
 }
@@ -782,24 +785,24 @@ D_set(void *ptr, PyObject *value, Py_ssize_t size)
 static PyObject *
 D_get(void *ptr, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(double complex)));
-    double complex x;
+    assert(NUM_BITS(size) || (size == 2*sizeof(double)));
+    double x[2];
 
     memcpy(&x, ptr, sizeof(x));
-    return PyComplex_FromDoubles(creal(x), cimag(x));
+    return PyComplex_FromDoubles(x[0], x[1]);
 }
 
 /* F: float complex */
 static PyObject *
 F_set(void *ptr, PyObject *value, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(float complex)));
+    assert(NUM_BITS(size) || (size == 2*sizeof(float)));
     Py_complex c = PyComplex_AsCComplex(value);
 
     if (c.real == -1 && PyErr_Occurred()) {
         return NULL;
     }
-    float complex x = CMPLXF((float)c.real, (float)c.imag);
+    float x[2] = {(float)c.real, (float)c.imag};
     memcpy(ptr, &x, sizeof(x));
     _RET(value);
 }
@@ -807,24 +810,24 @@ F_set(void *ptr, PyObject *value, Py_ssize_t size)
 static PyObject *
 F_get(void *ptr, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(float complex)));
-    float complex x;
+    assert(NUM_BITS(size) || (size == 2*sizeof(float)));
+    float x[2];
 
     memcpy(&x, ptr, sizeof(x));
-    return PyComplex_FromDoubles(crealf(x), cimagf(x));
+    return PyComplex_FromDoubles(x[0], x[1]);
 }
 
 /* G: long double complex */
 static PyObject *
 G_set(void *ptr, PyObject *value, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(long double complex)));
+    assert(NUM_BITS(size) || (size == 2*sizeof(long double)));
     Py_complex c = PyComplex_AsCComplex(value);
 
     if (c.real == -1 && PyErr_Occurred()) {
         return NULL;
     }
-    long double complex x = CMPLXL(c.real, c.imag);
+    long double x[2] = {c.real, c.imag};
     memcpy(ptr, &x, sizeof(x));
     _RET(value);
 }
@@ -832,11 +835,11 @@ G_set(void *ptr, PyObject *value, Py_ssize_t size)
 static PyObject *
 G_get(void *ptr, Py_ssize_t size)
 {
-    assert(NUM_BITS(size) || (size == sizeof(long double complex)));
-    long double complex x;
+    assert(NUM_BITS(size) || (size == 2*sizeof(long double)));
+    long double x[2];
 
     memcpy(&x, ptr, sizeof(x));
-    return PyComplex_FromDoubles((double)creall(x), (double)cimagl(x));
+    return PyComplex_FromDoubles((double)x[0], (double)x[1]);
 }
 #endif
 
@@ -1596,7 +1599,7 @@ for base_code, base_c_type in [
     ///////////////////////////////////////////////////////////////////////////
 
     TABLE_ENTRY_SW(d, &ffi_type_double);
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
+#if defined(Py_FFI_SUPPORT_C_COMPLEX)
     if (Py_FFI_COMPLEX_AVAILABLE) {
         TABLE_ENTRY(D, &ffi_type_complex_double);
         TABLE_ENTRY(F, &ffi_type_complex_float);
diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h
index 3b6d390728a07f..51ae71f7c6ef1e 100644
--- a/Modules/_ctypes/ctypes.h
+++ b/Modules/_ctypes/ctypes.h
@@ -11,8 +11,7 @@
 
 // Do we support C99 complex types in ffi?
 // For Apple's libffi, this must be determined at runtime (see gh-128156).
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-#   include "../_complex.h"       // complex
+#if defined(Py_FFI_SUPPORT_C_COMPLEX)
 #   if USING_APPLE_OS_LIBFFI && defined(__has_builtin)
 #       if __has_builtin(__builtin_available)
 #           define Py_FFI_COMPLEX_AVAILABLE __builtin_available(macOS 10.15, *)
@@ -493,11 +492,9 @@ struct tagPyCArgObject {
         double d;
         float f;
         void *p;
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
-        double complex D;
-        float complex F;
-        long double complex G;
-#endif
+        double D[2];
+        float F[2];
+        long double G[2];
     } value;
     PyObject *obj;
     Py_ssize_t size; /* for the 'V' tag */
diff --git a/Modules/_struct.c b/Modules/_struct.c
index e400f607b558dc..c36079f1eb8886 100644
--- a/Modules/_struct.c
+++ b/Modules/_struct.c
@@ -853,8 +853,8 @@ static const formatdef native_table[] = {
     {'e',       sizeof(short),  _Alignof(short),    nu_halffloat,   np_halffloat},
     {'f',       sizeof(float),  _Alignof(float),    nu_float,       np_float},
     {'d',       sizeof(double), _Alignof(double),   nu_double,      np_double},
-    {'F',       2*sizeof(float), _Alignof(float[2]), nu_float_complex, np_float_complex},
-    {'D',       2*sizeof(double), _Alignof(double[2]), nu_double_complex, np_double_complex},
+    {'F',       2*sizeof(float), _Alignof(float), nu_float_complex, np_float_complex},
+    {'D',       2*sizeof(double), _Alignof(double), nu_double_complex, np_double_complex},
     {'P',       sizeof(void *), _Alignof(void *),   nu_void_p,      np_void_p},
     {0}
 };


Reviews:
user: login: dalcinl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T11:13:29Z
user: login: skirpichev
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T11:20:25Z
user: login: skirpichev
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T11:24:55Z
user: login: dalcinl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T12:30:53Z
user: login: encukou
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T13:33:49Z
user: login: skirpichev
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T13:39:12Z
user: login: dalcinl
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T14:26:16Z
user: login: StanFromIreland
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T17:15:05Z
user: login: encukou
body: LGTM, merge if the buildbots don't complain.

state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-02T11:32:43Z
user: login: DangNhutNguyen
state: APPROVED
author_association: NONE
submitted_at: 2025-05-02T13:37:16Z
user: login: DangNhutNguyen
state: APPROVED
author_association: NONE
submitted_at: 2025-05-02T13:37:27Z
user: login: encukou
body: Looks good, thank you!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-05T09:20:34Z

Review comments:
diff_hunk: @@ -367,6 +360,12 @@ Notes:
    For the ``'E'`` and ``'C'`` format characters, the packed representation uses
path: Doc/library/struct.rst
user: login: dalcinl
body: Some PR has to update E and C in this line, but I'm not sure which one is doing it.  Just making sure you did not forget.
created_at: 2025-05-01T11:13:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -367,6 +360,12 @@ Notes:
    For the ``'E'`` and ``'C'`` format characters, the packed representation uses
path: Doc/library/struct.rst
user: login: skirpichev
body: Thanks, this was missed in #132827.
created_at: 2025-05-01T11:20:25Z
author_association: MEMBER
diff_hunk: @@ -367,6 +360,12 @@ Notes:
    For the ``'E'`` and ``'C'`` format characters, the packed representation uses
path: Doc/library/struct.rst
user: login: skirpichev
body: https://github.com/python/cpython/pull/133249
created_at: 2025-05-01T11:24:55Z
author_association: MEMBER
diff_hunk: @@ -878,13 +853,8 @@ static const formatdef native_table[] = {
     {'e',       sizeof(short),  _Alignof(short),    nu_halffloat,   np_halffloat},
     {'f',       sizeof(float),  _Alignof(float),    nu_float,       np_float},
     {'d',       sizeof(double), _Alignof(double),   nu_double,      np_double},
-#ifdef Py_HAVE_C_COMPLEX
-    {'F',       sizeof(float complex), _Alignof(float complex), nu_float_complex, np_float_complex},
-    {'D',       sizeof(double complex), _Alignof(double complex), nu_double_complex, np_double_complex},
-#else
-    {'F',       1, 0, nu_complex_stub, np_complex_stub},
-    {'D',       1, 0, nu_complex_stub, np_complex_stub},
-#endif
+    {'F',       2*sizeof(float), _Alignof(float[2]), nu_float_complex, np_float_complex},
+    {'D',       2*sizeof(double), _Alignof(double[2]), nu_double_complex, np_double_complex},
path: Modules/_struct.c
user: login: dalcinl
body: `_Alignof(float/double)` should be enough, right?

```suggestion
    {'F',       2*sizeof(float), _Alignof(float), nu_float_complex, np_float_complex},
    {'D',       2*sizeof(double), _Alignof(double), nu_double_complex, np_double_complex},
```
created_at: 2025-05-01T12:30:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -367,6 +360,12 @@ Notes:
    For the ``'E'`` and ``'C'`` format characters, the packed representation uses
    the IEEE 754 binary32 and binary64 format for components of the complex
    number, regardless of the floating-point format used by the platform.
+   Note that support for complex types is available unconditionally,
+   regardless on support IEC 60559 compatible complex arithmetic (Annex G of
+   the C11 standard) by the compiler.  Each complex type is interpreted as an
+   array type containing exactly two elements of the corresponding real type
+   (real and imaginary parts, respectively).
+
path: Doc/library/struct.rst
user: login: encukou
body: I'd not talk about Annex G at all. A compiler can support `_Complex` types but not Annex G (i.e. *IEC 60559-compatible* arithmetic).

I recommend  different wording from the standard, since terms like *type* and *array* have different meanings here.
```suggestion
   Note that complex types (``F`` and ``D``) are available unconditionally,
   despite complex types being an optional feature in C.
   As specified in the C11 standard, each complex type is represented by a
   two-element C array containing, respectively, the real and imaginary parts.

```
created_at: 2025-05-01T13:06:59Z
author_association: MEMBER
diff_hunk: @@ -763,80 +759,87 @@ d_get(void *ptr, Py_ssize_t size)
     return PyFloat_FromDouble(val);
 }
 
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
+#if defined(Py_FFI_SUPPORT_C_COMPLEX)
+
+/* We don't use Annex G complex types here, using arrays instead, as the C17+
path: Modules/_ctypes/cfield.c
user: login: encukou
body: ```suggestion
/* We don't use _Complex types here, using arrays instead, as the C17+
```
created_at: 2025-05-01T13:13:13Z
author_association: MEMBER
diff_hunk: @@ -763,80 +759,87 @@ d_get(void *ptr, Py_ssize_t size)
     return PyFloat_FromDouble(val);
 }
 
-#if defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)
+#if defined(Py_FFI_SUPPORT_C_COMPLEX)
+
+/* We don't use Annex G complex types here, using arrays instead, as the C17+
path: Modules/_ctypes/cfield.c
user: login: skirpichev
body: ```suggestion
/* We don't use _Complex types here, using arrays instead, as the C11+
```
created_at: 2025-05-01T13:39:11Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,2 @@
+Support the :c:expr:`float complex` and :c:expr:`double complex` C types in
path: Misc/NEWS.d/next/Library/2025-04-25-16-20-49.gh-issue-121249.uue2nK.rst
user: login: StanFromIreland
body: ```suggestion
Always support the :c:expr:`float complex` and :c:expr:`double complex` C types in
```
Would be more correct without delving into detail.
created_at: 2025-05-01T17:15:04Z
author_association: MEMBER

PR comments:
user: login: skirpichev
created_at: 2025-05-01T04:30:16Z
body: This is on top of https://github.com/python/cpython/pull/132864

CC @dalcinl
author_association: MEMBER
user: login: dalcinl
created_at: 2025-05-01T11:22:46Z
body: @skirpichev Thanks, it looks great. IMHO, and despite your doubts about worthiness, this should be merged. Things are simpler now, you removed lots of code, and a private header.

~Minor nit: I think the complex types logic that got added to [configure.ac](https://github.com/python/cpython/blob/main/configure.ac#L3835) can be safely removed.~ *Needed for ctypes tests.*
author_association: CONTRIBUTOR
user: login: bedevere-bot
created_at: 2025-05-02T11:30:20Z
body: :robot: New build scheduled with the buildbot fleet by @encukou for commit 071d57e8cfcab70c07aa7107841f7d8f1c5e89a5 :robot:

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133237%2Fmerge

If you want to schedule another build, you need to add the <kbd>:hammer: test-with-buildbots</kbd> label again.

author_association: NONE
user: login: skirpichev
created_at: 2025-05-02T13:28:59Z
body: @encukou, this pr was on top of #132864, which already tested with buildbots.  If we are going to merge this, I would prefer to do it via two separate commits.
author_association: MEMBER
user: login: encukou
created_at: 2025-05-02T13:34:32Z
body: Ah, I missed that note, sorry!
Could you add the latest wording changes there?
author_association: MEMBER
user: login: skirpichev
created_at: 2025-05-02T13:50:54Z
body: > Could you add the latest wording changes there?

Done.  See also minor docs correction in #133249
author_association: MEMBER
user: login: skirpichev
created_at: 2025-05-02T14:35:11Z
body: Failure in test_frame.py on s390x seems unrelated.
author_association: MEMBER
user: login: skirpichev
created_at: 2025-05-04T04:53:07Z
body: All failures are related to test_frame.py.

@encukou, I left here only one change, related to the struct module (cosmetic change in `_Alignof` operands), as it was related to code.  I hope it's ok.
author_association: MEMBER
user: login: skirpichev
created_at: 2025-05-04T04:55:50Z
body: @dalcinl, let us know if you think that linked issue should stay open after merging this.
author_association: MEMBER
user: login: dalcinl
created_at: 2025-05-04T06:03:26Z
body: @skirpichev I think you can close it. AFAICT, all concerns have been addressed. Many thanks for your hard work in implementing all these changes. 
author_association: CONTRIBUTOR