Repository: cython/cython

Issue:
[BUG] 'except?' started failing in 3.0.10
### Describe the bug

Usage of `except? -1` has seemingly started failing in 3.0.10. It works in all previous >3.0.0 versions.

Every place I use the "?"-version of `except`, it fails during `cythonize` with:
```
pyboy/utils.pxd:22:29: C method 'read_64bit' is declared but not defined
```

Note, it says "C method", but this is purely a Python/Cython function, it's not defining an external library.

### Code to reproduce the behaviour:

I define a `e.pxd` file with:
```cython
from libc.stdint cimport uint64_t

cdef class IntIOInterface:
    cpdef uint64_t read_64bit(self) except? -1
```

And a `e.py` file with:
```python
class IntIOInterface:

    def read_64bit(self):
        a = self.read()
        return a
```

Compile using following command:
```
cython --directive legacy_implicit_noexcept=True -3 e.py
```
### Expected behaviour

Expected to compile cleanly, and use `-1` to signal an exception like in previous versions.

### OS

_No response_

### Python version

_No response_

### Cython version

3.0.10

### Additional context

The code can be found here: https://github.com/Baekalfen/PyBoy/blob/ef25b6bb6445b6228069df57297620b0fe9487d8/pyboy/utils.pxd#L17

This bug report and fix is listed for Cython 3.0.10: https://github.com/cython/cython/issues/5709


Discussion:
I can't reproduce based on what you post here. I get

```
Error compiling Cython file:
------------------------------------------------------------
...
cpdef uint64_t read_64bit(self) except? -1
      ^
------------------------------------------------------------

e.pxd:1:6: 'uint64_t' is not a type identifier
```

Pretty irrespective of the Cython version.

If I add the line `from libc.stdint cimport uint64_t` then it compiles for me.

#5709 and #5710 shouldn't be relevant since they're only in the master branch and not a Cython release


Sorry for being too vague. You're right.

I narrowed it down, and it only coincides with `legacy_implicit_noexcept`:
```sh
$ cython e.py --directive legacy_implicit_noexcept=True
/opt/homebrew/lib/python3.11/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /Users/mads/Desktop/cytest/e.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
warning: e.py:5:4: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
warning: e.py:5:4: Compatible but non-identical C method 'read_64bit' not redeclared in definition part of extension type 'IntIOInterface'.  This may cause incorrect vtables to be generated.
warning: e.pxd:6:29: Previous declaration is here

Error compiling Cython file:
------------------------------------------------------------
...
cimport cython
from libc.stdint cimport int64_t, uint8_t, uint16_t, uint32_t, uint64_t


cdef class IntIOInterface:
    cpdef uint64_t read_64bit(self) except? -1                             ^
------------------------------------------------------------

e.pxd:6:29: C method 'read_64bit' is declared but not defined
```
And you're right about #5709 and #5710, I mistook 3.1.0 for 3.0.10 in the changelog https://cython.readthedocs.io/en/latest/src/changes.html
Right - this looks to bisect to 4e9f7307312881111b6f56279a83812a2309cd16 (which is probably unsurprising)
Was it supposed to do that, or do you still think it's a bug?
It's a bug.

I think it moved a check earlier before it's applied the typing from the pxd file.

That probably suggests Cython doesn't have sufficient tests for `legacy_implicit_noexcept` and typing applied from pxd files.
> I can't reproduce based on what you post here. I get
> 
> ```
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cpdef uint64_t read_64bit(self) except? -1
>       ^
> ------------------------------------------------------------
> 
> e.pxd:1:6: 'uint64_t' is not a type identifier
> ```
> 
> Pretty irrespective of the Cython version.
> 
> If I add the line `from libc.stdint cimport uint64_t` then it compiles for me.
> 
> #5709 and #5710 shouldn't be relevant since they're only in the master branch and not a Cython release

I am not able to reproduce it too. I tried following:

* e.pxd:

```pyx
from libc.stdint cimport uint64_t

cpdef uint64_t read_64bit(self) except? -1
```
* e.py:
```py
def read_64bit(self):
        a = self.read()
        return a

```

I compiled it using:

```
cython --directive legacy_implicit_noexcept=True -3 e.py
```

but I did not get any error... (just warning about `noexcept`). Am I doing something wrong?
> but I did not get any error... (just warning about `noexcept`). Am I doing something wrong?

Could you double check, that you're on Cython 3.0.10?
```
% cat e.py
def read_64bit(self):
        a = self.read()
        return a

% cat e.pxd
from libc.stdint cimport uint64_t

cpdef uint64_t read_64bit(self) except? -1

% git status
HEAD detached at 3.0.10
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	./

nothing added to commit but untracked files present (use "git add" to track)

% cython -V
Cython version 3.0.10

% cython --directive legacy_implicit_noexcept=True -3 e.py
warning: e.py:1:0: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
```
@matusvalo it may need to be in a `cdef class`. I modified it to be like that when I tested it but I'm not sure if it's a necessary part of the issue
Confirmed, it has to be in a class.

New pxd:
```cython
cimport cython
from libc.stdint cimport int64_t, uint8_t, uint16_t, uint32_t, uint64_t


cdef class IntIOInterface:
    cpdef uint64_t read_64bit(self) except? -1
```

New py:
```python
class IntIOInterface:
    def __init__(self, buf):
        pass

    def read_64bit(self):
        a = self.read()
        b = self.read()
        c = self.read()
        d = self.read()
        e = self.read()
        f = self.read()
        g = self.read()
        h = self.read()
        return a | (b << 8) | (c << 16) | (d << 24) | (e << 32) | (f << 40) | (g << 48) | (h << 56)
```
I confirm.
Let me update the description to contain reproducible example.
I think possible solution is to replace line:

https://github.com/cython/cython/blob/9ba08c73193fd850ea4465e1928bdc40c60dae0e/Cython/Compiler/Nodes.py#L714

with following:

```python
            if not (self.declared_name() in env.entries and not in_pxd):
                self.exception_check = False
```

The reason is because when .pxd file is defined there are two runs `CFuncDeclaratorNode.analyse`:
  1. run is executed for .pxd file and is triggered by the parser
  2. run is executed for .py file and is triggered by `ParseTreeTransform.AlignFunctionDefinitions.visit_DefNode`

The problem is causing 2nd run because it does uses information from entry which misses correct value of `has_explicit_exc_clause` (default value `False` is used).

Possible solution (shown above) is to not override `exception_check` when `CFuncDeclaratorNode` is already declared by .pxd file since it already contains correct value of `exception_check` from 1st run. Other solution would be to add `has_explicit_exc_clause` information to `CFuncType` which I don't like.

Maybe there is some other solution but currently I don't see any.
> I think possible solution is to replace line:

That looks reasons based on a quick look.
> I think possible solution is to replace line:
> 
> https://github.com/cython/cython/blob/9ba08c73193fd850ea4465e1928bdc40c60dae0e/Cython/Compiler/Nodes.py#L714
> 
> with following:
> 
> ```python
>             if not (self.declared_name() in env.entries and not in_pxd):
>                 self.exception_check = False
> ```

This fixes the issue on my end. Can't comment on the soundness internally in Cython.

Resolving patch:
diff --git a/Cython/Compiler/Nodes.py b/Cython/Compiler/Nodes.py
--- a/Cython/Compiler/Nodes.py
+++ b/Cython/Compiler/Nodes.py
@@ -710,8 +710,10 @@ def analyse(self, return_type, env, nonempty=0, directive_locals=None, visibilit
                 and not self.has_explicit_exc_clause
                 and self.exception_check
                 and visibility != 'extern'):
+            # If function is already declared from pxd, the exception_check has already correct value.
+            if not (self.declared_name() in env.entries and not in_pxd):
+                self.exception_check = False
             # implicit noexcept, with a warning
-            self.exception_check = False
             warning(self.pos,
                     "Implicit noexcept declaration is deprecated."
                     " Function declaration should contain 'noexcept' keyword.",


Test patch:
diff --git a/tests/run/legacy_implicit_noexcept_build.srctree b/tests/run/legacy_implicit_noexcept_build.srctree
--- a/tests/run/legacy_implicit_noexcept_build.srctree
+++ b/tests/run/legacy_implicit_noexcept_build.srctree
@@ -1,5 +1,7 @@
 PYTHON setup.py build_ext --inplace
 PYTHON -c "import bar"
+PYTHON -c "import run"
+PYTHON -c "import crun"
 
 ######## setup.py ########
 
@@ -7,9 +9,175 @@ from Cython.Build.Dependencies import cythonize
 from distutils.core import setup
 
 setup(
-  ext_modules = cythonize("*.pyx", compiler_directives={'legacy_implicit_noexcept': True}),
+  ext_modules = cythonize(["*.pyx", "aa.py"], compiler_directives={'legacy_implicit_noexcept': True}),
 )
 
+######## run.py ########
+
+import aa
+
+def check_exception_propagated(c, ret_expected):
+    assert c(False) == ret_expected
+    try:
+        c(True)
+    except ValueError:
+        pass
+    else:
+        assert False, "ValueError not raised"
+
+def check_exception_not_propagated(c, ret_expected):
+    assert c(False) == ret_expected
+    try:
+        c(True)
+    except ValueError:
+        assert False, "ValueError raised"
+    else:
+        pass
+
+klass = aa.Klass()
+
+check_exception_propagated(klass.m1, 1)
+check_exception_not_propagated(klass.m2, 2)
+check_exception_not_propagated(klass.m3, 3)
+check_exception_propagated(klass.m4, 4)
+check_exception_propagated(klass.m5, 5)
+
+check_exception_propagated(klass.n1, "n1")
+check_exception_propagated(klass.n2, "n2")
+check_exception_propagated(klass.n3, "n3")
+
+check_exception_not_propagated(klass.o3, None)
+check_exception_propagated(klass.o2, None)
+check_exception_not_propagated(klass.o3, None)
+
+check_exception_propagated(aa.m1, 1)
+check_exception_not_propagated(aa.m2, 2)
+check_exception_not_propagated(aa.m3, 3)
+check_exception_propagated(aa.m4, 4)
+check_exception_propagated(aa.m5, 5)
+
+check_exception_propagated(aa.n1, "n1")
+check_exception_propagated(aa.n2, "n2")
+check_exception_propagated(aa.n3, "n3")
+
+check_exception_not_propagated(aa.o3, None)
+check_exception_propagated(aa.o2, None)
+check_exception_not_propagated(aa.o3, None)
+
+######## crun.pyx ########
+
+cimport aa
+cdef aa.Klass klass = aa.Klass()
+
+assert klass.p1(False) == 1
+try:   klass.p1(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert klass.p2(False) == 2
+try:   klass.p2(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+assert klass.p3(False) == 3
+try:   klass.p3(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+assert klass.p4(False) == 4
+try:   klass.p4(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert klass.p5(False) == 5
+try:   klass.p5(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert klass.q1(False) == 'q1'
+try:   klass.q1(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert klass.q2(False) == 'q2'
+try:   klass.q2(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert klass.q3(False) == 'q3'
+try:   klass.q3(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+klass.r1(False)
+try: klass.r1(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+klass.r2(False)
+try: klass.r2(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+klass.r3(False)
+try: klass.r3(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+assert aa.p1(False) == 1
+try: aa.p1(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert aa.p2(False) == 2
+try: aa.p2(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+assert aa.p3(False) == 3
+try: aa.p3(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+assert aa.p4(False) == 4
+try:   aa.p4(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert aa.p5(False) == 5
+try:   aa.p5(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert aa.q1(False) == 'q1'
+try:   aa.q1(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert aa.q2(False) == 'q2'
+try:   aa.q2(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+assert aa.q3(False) == 'q3'
+try:   aa.q3(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+aa.r1(False)
+try: aa.r1(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
+
+aa.r2(False)
+try: aa.r2(True)
+except ValueError: pass
+else:              assert False, "ValueError not raised"
+
+aa.r3(False)
+try: aa.r3(True)
+except ValueError: assert False, "ValueError not raised"
+else:              pass
 
 ######## bar.pyx ########
 
@@ -31,3 +199,202 @@ except RuntimeError:
     pass
 else:
     assert False, 'Exception not raised'
+
+
+######## aa.pxd ########
+cdef class Klass:
+    cpdef int m1(self, bint exc) except? -1
+    cpdef int m2(self, bint exc)
+    cpdef int m3(self, bint exc) noexcept
+    cpdef int m4(self, bint exc) except *
+    cpdef int m5(self, bint exc) except -1
+    cpdef object n1(self, bint exc)
+    cpdef object n2(self, bint exc) except *
+    cpdef object n3(self, bint exc) noexcept  # will warn, but should work
+    cpdef void o1(self, bint exc)
+    cpdef void o2(self, bint exc) except*
+    cpdef void o3(self, bint exc) noexcept
+    cdef int p1(self, bint exc) except? -1
+    cdef int p2(self, bint exc)
+    cdef int p3(self, bint exc) noexcept
+    cdef int p4(self, bint exc) except *
+    cdef int p5(self, bint exc) except -1
+    cdef object q1(self, bint exc)
+    cdef object q2(self, bint exc) except *
+    cdef object q3(self, bint exc) noexcept  # will warn, but should work
+    cdef void r1(self, bint exc)
+    cdef void r2(self, bint exc) except*
+    cdef void r3(self, bint exc) noexcept
+
+cpdef int m1(bint exc) except? -1
+cpdef int m2(bint exc)
+cpdef int m3(bint exc) noexcept
+cpdef int m4(bint exc) except *
+cpdef int m5(bint exc) except -1
+cpdef object n1(bint exc)
+cpdef object n2(bint exc) except *
+cpdef object n3(bint exc) noexcept  # will warn, but should work
+cpdef void o1(bint exc)
+cpdef void o2(bint exc) except*
+cpdef void o3(bint exc) noexcept
+cdef int p1(bint exc) except? -1
+cdef int p2(bint exc)
+cdef int p3(bint exc) noexcept
+cdef int p4(bint exc) except *
+cdef int p5(bint exc) except -1
+cdef object q1(bint exc)
+cdef object q2(bint exc) except *
+cdef object q3(bint exc) noexcept  # will warn, but should work
+cdef void r1(bint exc)
+cdef void r2(bint exc) except*
+cdef void r3(bint exc) noexcept
+
+######## aa.py ########
+
+class Klass:
+
+    def m1(self, exc):
+        if exc: raise ValueError()
+        return 1
+    def m2(self, exc):
+        print('m2 1', exc)
+        if exc: raise ValueError()
+        print('m2 2', exc)
+        return 2
+    def m3(self, exc):
+        if exc: raise ValueError()
+        return 3
+    def m4(self, exc):
+        if exc: raise ValueError()
+        return 4
+    def m5(self, exc):
+        if exc: raise ValueError()
+        return 5
+
+    def n1(self, exc):
+        if exc: raise ValueError()
+        return 'n1'
+    def n2(self, exc):
+        if exc: raise ValueError()
+        return 'n2'
+    def n3(self, exc):
+        if exc: raise ValueError()
+        return 'n3'
+
+    def o1(self, exc):
+        if exc: raise ValueError()
+        return
+    def o2(self, exc):
+        if exc: raise ValueError()
+        return
+    def o3(self, exc):
+        if exc: raise ValueError()
+        return
+
+    def p1(self, exc):
+        if exc: raise ValueError()
+        return 1
+    def p2(self, exc):
+        if exc: raise ValueError()
+        return 2
+    def p3(self, exc):
+        if exc: raise ValueError()
+        return 3
+    def p4(self, exc):
+        if exc: raise ValueError()
+        return 4
+    def p5(self, exc):
+        if exc: raise ValueError()
+        return 5
+
+    def q1(self, exc):
+        if exc: raise ValueError()
+        return 'q1'
+    def q2(self, exc):
+        if exc: raise ValueError()
+        return 'q2'
+    def q3(self, exc):
+        if exc: raise ValueError()
+        return 'q3'
+
+    def r1(self, exc):
+        if exc: raise ValueError()
+        return
+    def r2(self, exc):
+        if exc: raise ValueError()
+        return
+    def r3(self, exc):
+        if exc: raise ValueError()
+        return
+
+def m1(exc):
+    if exc: raise ValueError()
+    return 1
+def m2(exc):
+    if exc: raise ValueError()
+    return 2
+def m3(exc):
+    if exc: raise ValueError()
+    return 3
+def m4(exc):
+    if exc: raise ValueError()
+    return 4
+def m5(exc):
+    if exc: raise ValueError()
+    return 5
+
+def n1(exc):
+    if exc: raise ValueError()
+    return 'n1'
+def n2(exc):
+    if exc: raise ValueError()
+    return 'n2'
+def n3(exc):
+    if exc: raise ValueError()
+    return 'n3'
+
+def o1(exc):
+    if exc: raise ValueError()
+    return
+def o2(exc):
+    if exc: raise ValueError()
+    return
+def o3(exc):
+    if exc: raise ValueError()
+    return
+
+def p1(exc):
+    if exc: raise ValueError()
+    return 1
+def p2(exc):
+    if exc: raise ValueError()
+    return 2
+def p3(exc):
+    if exc: raise ValueError()
+    return 3
+def p4(exc):
+    if exc: raise ValueError()
+    return 4
+def p5(exc):
+    if exc: raise ValueError()
+    return 5
+
+def q1(exc):
+    if exc: raise ValueError()
+    return 'q1'
+def q2(exc):
+    if exc: raise ValueError()
+    return 'q2'
+def q3(exc):
+    if exc: raise ValueError()
+    return 'q3'
+
+def r1(exc):
+    if exc: raise ValueError()
+    return
+def r2(exc):
+    if exc: raise ValueError()
+    return
+def r3(exc):
+    if exc: raise ValueError()
+    return