Repository: cython/cython

Issue:
'final' subclasses produce suboptimal self casts
Starting from:
```cython
# subclass.pyx
cdef class A:
    cdef int foo(self):
        return -1

    cdef int bar(self, int x):
        return -1

cdef class Sub(A):
    cdef int foo(self):
        return self.bar(0)

    cdef int bar(self, int x):
        return x + 1
```
compiles fine with `cythonize -b subclass.pyx`. If you add `@cython.final` onto `Sub`, then it produces a non-fatal warning:
```
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -m64 -fPIC -m64 -fPIC -fPIC -Iconda/envs/cp36/include/python3.6m -c subclass.c -o build/temp.linux-x86_64-3.6/subclass.o
subclass.c: In function ‘__pyx_f_8subclass_3Sub_foo’:
subclass.c:1667:41: warning: passing argument 1 of ‘__pyx_f_8subclass_3Sub_bar’ from incompatible pointer type [-Wincompatible-pointer-types]
   __pyx_r = __pyx_f_8subclass_3Sub_bar(((struct __pyx_obj_8subclass_A *)__pyx_v_self), 0);
                                        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
subclass.c:1205:85: note: expected ‘struct __pyx_obj_8subclass_Sub *’ but argument is of type ‘struct __pyx_obj_8subclass_A *’
 static int __pyx_f_8subclass_3Sub_bar(CYTHON_UNUSED struct __pyx_obj_8subclass_Sub *__pyx_v_self, int __pyx_v_x); /* proto*/
                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
```
But if you _also_ add `# distutils: language = c++` to the file to compile as C++, then it fails to compile:
```
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -m64 -fPIC -m64 -fPIC -fPIC -Iconda/envs/cp36/include/python3.6m -c subclass.cpp -o build/temp.linux-x86_64-3.6/subclass.o
subclass.cpp: In function ‘int __pyx_f_8subclass_3Sub_foo(__pyx_obj_8subclass_Sub*)’:
subclass.cpp:1682:41: error: cannot convert ‘__pyx_obj_8subclass_A*’ to ‘__pyx_obj_8subclass_Sub*’
   __pyx_r = __pyx_f_8subclass_3Sub_bar(((struct __pyx_obj_8subclass_A *)__pyx_v_self), 0);
                                        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
subclass.cpp:1220:85: note:   initializing argument 1 of ‘int __pyx_f_8subclass_3Sub_bar(__pyx_obj_8subclass_Sub*, int)’
 static int __pyx_f_8subclass_3Sub_bar(CYTHON_UNUSED struct __pyx_obj_8subclass_Sub *__pyx_v_self, int __pyx_v_x); /* proto*/
                                                                                     ^
error: command 'gcc' failed with exit status 1
```


Discussion:
Note that in the C case (which produces just a warning), it seems to work correctly. That is, if we add:
```cython
def do_foo():
    cdef A a 

    a = Sub()
    return a.foo()
```
and call it, it returns 1 whether there's the `cython.final` decorator or not.
Thanks for the report. Indeed, the code is correct, but the C compiler does not understand the subtype relationship among these pointers (hence the warning and error).
Such compilation warnings are also observed when building scikit-learn (see [this trace](https://gist.github.com/jjerphan/8cfbb5349f5680856460ff80152ab69d#file-full-trace-log-L492-L696)) for Cython extension types hierarchy whose final extension types aren't necessarily decorated with `@final`.

IIRC, if we switch to a C++ compiler for those unit of compilation (e.g. using `language = c++`), then we have a fatal compilation error.

Still, we do not have any problems at runtime.
We might be able to mitigate this by generating a more suitable cast, at least in some cases.
As noted at https://bugs.launchpad.net/lxml/+bug/2045435/comments/1 (just commenting here so it's easier to search for), this becomes a problem with GCC 14 which makes `-Wincompatible-pointer-types` an error by default.
Yes, I don't understand the title change. It's not suboptimal, it's outright not able to compile (in C++ mode).
And it won't be able to in C in April either.
What's failing here is that we cast the `self` argument to the inherited type of the original `self` argument defining the method, not the `self` of the final C function of the concrete subtype that we actually call.

https://github.com/cython/cython/blob/f2d564a06b775bd1cedfc36f5fc8996865ad3675/Cython/Compiler/ExprNodes.py#L6333

It feels like we should actually know the type of the concrete C function implementation, but it seems that we lose it when inheriting the original type. For calling `final` methods, we separately remember the `final_func_cname` of the C function and then call that. However, I think we should instead create a real entry for them to remember name and type, and then use that directly when analysing the method attribute in `AttributeNode`. `final` methods might not be more special than that, after all.
I was wondering if as an immediate workaround for the failures we see in Fedora with GCC 14, we could add:

```
#pragma GCC diagnostic ignored "-Wincompatible-pointer-types"
...
#pragma GCC diagnostic pop
```
 - Gentoo: https://github.com/gentoo/gentoo/pull/35549
 - Fedora: https://src.fedoraproject.org/rpms/Cython/pull-request/57

Should we temporarily add this to Cython upstream?
For the record, my patch is pretty poor. It would certainly be better to limit the scope (i.e. `push`/`pop`) to the call itself.
If this is the best we currently have, then we can add the pragma for the time being. It works around a real user problem. But we should limit it to the absolute minimum of code, so that users still benefit from the C compiler diagnostics for their own code.
See https://github.com/cython/cython/pull/6039
@scoder Would it be possible to cut a point release soon with the workaround? GCC 14 is fast approaching and it'd be great to not have to keep telling other folks about this gotcha. Many thanks.

Resolving patch:
diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py
+++ b/Cython/Compiler/ExprNodes.py
@@ -6521,18 +6521,7 @@ def generate_result_code(self, code):
                         goto_error = code.error_goto_if(" && ".join(exc_checks), self.pos)
                     else:
                         goto_error = ""
-                    undo_gh2747_hack = False
-                    if (self.function.is_attribute and
-                            self.function.entry and self.function.entry.final_func_cname and
-                            not code.funcstate.scope.is_cpp()
-                            ):
-                        # Hack around https://github.com/cython/cython/issues/2747
-                        # Disable GCC warnings/errors for wrong self casts.
-                        code.putln('#pragma GCC diagnostic ignored "-Wincompatible-pointer-types"')
-                        undo_gh2747_hack = True
                     code.putln("%s%s; %s" % (lhs, rhs, goto_error))
-                    if undo_gh2747_hack:
-                        code.putln("#pragma GCC diagnostic pop")
                 if self.type.is_pyobject and self.result():
                     self.generate_gotref(code)
             if self.has_optional_args:
diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
--- a/Cython/Compiler/ModuleNode.py
+++ b/Cython/Compiler/ModuleNode.py
@@ -3935,7 +3935,8 @@ def generate_exttype_vtable_init_code(self, entry, code):
                 if entry.func_cname]
             if c_method_entries:
                 for meth_entry in c_method_entries:
-                    cast = meth_entry.type.signature_cast_string()
+                    vtable_type = meth_entry.vtable_type or meth_entry.type
+                    cast = vtable_type.signature_cast_string()
                     code.putln(
                         "%s.%s = %s%s;" % (
                             type.vtable_cname,
diff --git a/Cython/Compiler/Symtab.py b/Cython/Compiler/Symtab.py
--- a/Cython/Compiler/Symtab.py
+++ b/Cython/Compiler/Symtab.py
@@ -242,6 +242,7 @@ class Entry(object):
     known_standard_library_import = None
     pytyping_modifiers = None
     enum_int_value = None
+    vtable_type = None
 
     def __init__(self, name, cname, type, pos = None, init = None):
         self.name = name
@@ -2551,6 +2552,9 @@ def declare_cfunction(self, name, type, pos,
         if self.parent_type.is_final_type or entry.is_inline_cmethod or self.directives.get('final'):
             entry.is_final_cmethod = True
             entry.final_func_cname = entry.func_cname
+            if not type.is_fused:
+                entry.vtable_type = entry.type
+                entry.type = type
 
         return entry
 


Test patch:
diff --git a/tests/bugs.txt b/tests/bugs.txt
--- a/tests/bugs.txt
+++ b/tests/bugs.txt
@@ -9,7 +9,6 @@ cfunc_call_tuple_args_T408
 generator_expressions_in_class
 for_from_pyvar_loop_T601
 temp_sideeffects_T654    # not really a bug, Cython warns about it
-inherited_final_method
 cimport_alias_subclass
 
 # PEP-489 is currently disabled
diff --git a/tests/run/inherited_final_method.pyx b/tests/run/inherited_final_method.pyx
--- a/tests/run/inherited_final_method.pyx
+++ b/tests/run/inherited_final_method.pyx
@@ -3,32 +3,73 @@
 
 cimport cython
 
+cdef class TopBase:
+    cdef method(self):
+        return None
+
 
-cdef class BaseClass:
+cdef class BaseClass(TopBase):
     """
     >>> obj = BaseClass()
     >>> obj.call_base()
-    True
+    'base'
     """
+    cdef base(self):
+        return '@base'
+
     cdef method(self):
-        return True
+        return 'base'
 
     def call_base(self):
         return self.method()
 
 
 @cython.final
-cdef class Child(BaseClass):
+cdef class ChildClass(BaseClass):
     """
-    >>> obj = Child()
+    >>> obj = ChildClass()
     >>> obj.call_base()
-    True
+    'child'
     >>> obj.call_child()
-    True
+    'child'
     """
+    cdef child(self):
+        return '@child'
+
     cdef method(self):
-        return True
+        return 'child'
 
     def call_child(self):
         # original bug: this requires a proper cast for self
         return self.method()
+
+
+def test_BaseClass():
+    """
+    >>> test_BaseClass()
+    '@base'
+    'base'
+    """
+    cdef BaseClass obj = BaseClass()
+    print(repr(obj.base()))
+    print(repr(obj.method()))
+
+
+def test_ChildClass():
+    """
+    >>> test_ChildClass()
+    '@base'
+    '@child'
+    'child'
+    '@base'
+    '@child'
+    'child'
+    """
+    cdef ChildClass obj = ChildClass()
+    print(repr(obj.base()))
+    print(repr(obj.child()))
+    print(repr(obj.method()))
+    cdef BaseClass bobj = obj
+    print(repr(obj.base()))
+    print(repr(obj.child()))
+    print(repr(bobj.method()))