Repository: cython/cython

Issue:
Support special binary methods ("__add__" etc.) as Python does
I'm not sure if this is a bug report or a feature request, but as of cython 0.27.3 `__radd__`, `__rtruediv__` and other right side magic methods do not work:
```python
In [1]: %load_ext Cython

In [2]: %%cython
   ...: cdef class Example:
   ...:   def __radd__(self, other):
   ...:     print('radd', self, other)
   ...:   def __rtruediv__(self, other):
   ...:     print('rtruediv', self, other)
   ...:

In [3]: 3 + Example()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-b54d8dd2451f> in <module>()
----> 1 3 + Example()

TypeError: unsupported operand type(s) for +: 'int' and '_cython_magic_c1fde2d95cc29b87ed98f696ccfef975.Example'

In [4]: 'foo' / Example()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-1e29a35d9b28> in <module>()
----> 1 'foo' / Example()

TypeError: unsupported operand type(s) for /: 'str' and '_cython_magic_c1fde2d95cc29b87ed98f696ccfef975.Example'

In [5]: import Cython

In [6]: Cython.__version__
Out[6]: '0.27.3'
```
I'm using the following workaround, but it looks ugly:
```python
In [7]: %%cython
   ...: cdef class Example:
   ...:   def __add__(other, self):
   ...:     print('add', self, other)
   ...:   def __truediv__(other, self):
   ...:     print('truediv', self, other)
   ...:

In [8]: 3 + Example()
add <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a0f0> 3

In [9]: 'foo' / Example()
truediv <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a110> foo
```
and doesn't really work if we switch operands:
```python
In [10]: Example() + 3
add 3 <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a110>

In [11]: Example() / 'foo'
truediv foo <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109468e60>
```

I'm not familiar with cython codebase, but a quick search through the source, issues and PRs doesn't show anything related.


Discussion:
Cython's special methods still mostly reflect the C API rather than the
Python API. See
http://cython.readthedocs.io/en/latest/src/userguide/special_methods.html#arithmetic-methods
. Here either the first or second parameter may be self, rather than having
__rxxx__ versions of the operators. (It would be nice to support this
however, similar to how the comparison methods were recently added.)

On Thu, Dec 28, 2017 at 3:14 AM, Dmitry Malinovsky <notifications@github.com
> wrote:

> I'm not sure if this is a bug report or a feature request, but as of
> cython 0.27.3 __radd__, __rtruediv__ and other right side magic methods
> do not work:
>
> In [1]: %load_ext Cython
>
> In [2]: %%cython
>    ...: cdef class Example:
>    ...:   def __radd__(self, other):
>    ...:     print('radd', self, other)
>    ...:   def __rtruediv__(self, other):
>    ...:     print('rtruediv', self, other)
>    ...:
>
> In [3]: 3 + Example()---------------------------------------------------------------------------TypeError                                 Traceback (most recent call last)<ipython-input-3-b54d8dd2451f> in <module>()----> 1 3 + Example()
> TypeError: unsupported operand type(s) for +: 'int' and '_cython_magic_c1fde2d95cc29b87ed98f696ccfef975.Example'
>
> In [4]: 'foo' / Example()---------------------------------------------------------------------------TypeError                                 Traceback (most recent call last)<ipython-input-4-1e29a35d9b28> in <module>()----> 1 'foo' / Example()
> TypeError: unsupported operand type(s) for /: 'str' and '_cython_magic_c1fde2d95cc29b87ed98f696ccfef975.Example'
>
> In [5]: import Cython
>
> In [6]: Cython.__version__
> Out[6]: '0.27.3'
>
> I'm using the following workaround, but it looks ugly:
>
> In [7]: %%cython
>    ...: cdef class Example:
>    ...:   def __add__(other, self):
>    ...:     print('add', self, other)
>    ...:   def __truediv__(other, self):
>    ...:     print('truediv', self, other)
>    ...:
>
> In [8]: 3 + Example()
> add <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a0f0> 3
>
> In [9]: 'foo' / Example()
> truediv <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a110> foo
>
> and doesn't really work if we switch operands:
>
> In [10]: Example() + 3
> add 3 <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109d7a110>
>
> In [11]: Example() / 'foo'
> truediv foo <_cython_magic_45c6eacd188f8181199b3c76f32cba5d.Example object at 0x109468e60>
>
> I'm not familiar with cython codebase, but a quick search through the
> source, issues and PRs doesn't show anything related.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/cython/cython/issues/2056>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAdqgcvFpRoV1V5VQmmG1gnKEIwYNzK5ks5tE3gWgaJpZM4ROO6o>
> .
>

We should consider whether we want to move away from the "first or second argument is self" for the arithmetic methods for 3.0.
I think we should switch to normal Python semantics. As @robertwb said, in the same way as we support the comparison methods now.
See [`generate_richcmp_function()`](https://github.com/cython/cython/blob/d034742afc7d9fc37a6f3eb24cda07aa86edb0ca/Cython/Compiler/ModuleNode.py#L1834-L1902) in `ModuleNode.py` together with the [`RichcmpSlot`](https://github.com/cython/cython/blob/d034742afc7d9fc37a6f3eb24cda07aa86edb0ca/Cython/Compiler/TypeSlots.py#L408-L416) in `TypeSlots.py`.

The idea is to define the number slots with a new slot type `BinopSlot` that has two special Python methods as implementation, and then generate the `nb_add` etc. slots by dispatching to these methods (more precisely, their C functions) according to CPython's own logic in [`typeobject.c`](https://github.com/python/cpython/blob/d929f1838a8fba881ff0148b7fc31f6265703e3d/Objects/typeobject.c#L6128-L6166).
Marking as Cython 3.0 release blocker, in case we need backwards incompatible changes for this.

Resolving patch:
diff --git a/Cython/Compiler/Options.py b/Cython/Compiler/Options.py
--- a/Cython/Compiler/Options.py
+++ b/Cython/Compiler/Options.py
@@ -178,7 +178,7 @@ def get_directive_defaults():
     'auto_pickle': None,
     'cdivision': False,  # was True before 0.12
     'cdivision_warnings': False,
-    'c_api_binop_methods': True,  # Change for 3.0
+    'c_api_binop_methods': False,  # was True before 3.0
     'overflowcheck': False,
     'overflowcheck.fold': True,
     'always_allow_keywords': True,


Test patch:
diff --git a/tests/run/unicode_formatting.pyx b/tests/run/unicode_formatting.pyx
--- a/tests/run/unicode_formatting.pyx
+++ b/tests/run/unicode_formatting.pyx
@@ -38,21 +38,21 @@ def mix_format(a, int b, list c):
 
 class PySubtype(unicode):
     def __rmod__(self, other):
-        return f'PyRMOD({other})'
+        return f'PyRMOD({self}, {other})'
 
 
 cdef class ExtSubtype(unicode):
-    def __mod__(one, other):
-        return f'ExtMOD({one}, {other})'
+    def __rmod__(self, other):
+        return f'ExtRMOD({self}, {other})'
 
 
 def subtypes():
     """
     >>> py, ext = subtypes()
     >>> print(py)
-    PyRMOD(-%s-)
+    PyRMOD(PySub, -%s-)
     >>> print(ext)
-    ExtMOD(-%s-, ExtSub)
+    ExtRMOD(ExtSub, -%s-)
     """
     return [
         '-%s-' % PySubtype("PySub"),