Repository: chainer/chainer

Issue:
Link.__call__ in v4.2.0 and v5.0.0b2 is not strictly backward-compatible
IIUC, to introduce `forward` interface while keeping backward-compatibility, `Link.__call__` is defined in #4770 #4888. But @tkanmae noticed that it isn't strictly backward-compatible.

The incompatibility arises from MRO's corner case. The following example illustrates the problem.

```python
import chainer
import numpy

class A(object):
    def __call__(self, x, y):
        return x + y

class MyLink(chainer.Link, A):
    pass

x = numpy.arange(10)
y = numpy.arange(10)
link = MyLink()
z = link(x, y)
print(z)
```

That said, most of such codes would be fallacious mix-in of `A`; in that case users should fix the code to the following.

```python
class MyLink(A, chainer.Link):
    pass
```

But there may be some cases such an inheritance are legitimate. Even if you think users should fix their codes, at least you should add a mention in the release notes.

Keeping backward-compatibility is also possible, whereas some metaprogramming hack is required. Following is the proof of concept. (Only tested in Python 3.5.)

```python
diff --git a/chainer/link.py b/chainer/link.py
index 7782f9e4f..2bd7adf40 100644
--- a/chainer/link.py
+++ b/chainer/link.py
@@ -198,8 +198,20 @@ class Link(object):
         try:
             forward = self.forward
         except AttributeError:
+            mro = type(self).__mro__
+            for link_idx, cls in enumerate(mro):
+                if cls is Link:
+                    break
+            class MroAfterMeta(type):
+                def mro(cls):
+                    return (cls,) + mro[link_idx + 1:]
+            class MroAfter(type(self), metaclass=MroAfterMeta):
+                def __init__(self):
+                    pass
+            if callable(MroAfter()):
+                return MroAfter.__call__(self, *args, **kwargs)
             raise TypeError(
-                '{} object has neither \'Link.__call__\' method overridden'
+                '{} object has neither \'__call__\' method defind'
                 ' nor \'forward\' method defined.'.format(self))
         return forward(*args, **kwargs)
 ```


Discussion:
Thank you for the report. It looks we should make a hot fix for this problem. I think the following change is enough.

```py
class Link(object):
    ...
    def __call__(self, *args, **kwargs):
        forward = getattr(super(Link, self), '__call__', None)
        if forward is None:
            forward = self.forward
        return forward(*args, **kwargs)
```
@beam2d Thanks. I agree with that your fix is enough. I misunderstood the behavior of `super`.

(I've checked the implementation of `super`, and found that it searches `self`'s MRO.  https://github.com/python/cpython/blob/v3.4.8/Objects/typeobject.c#L6946)



Resolving patch:
diff --git a/chainer/link.py b/chainer/link.py
--- a/chainer/link.py
+++ b/chainer/link.py
@@ -194,7 +194,13 @@ def __init__(self):
             self._within_init_scope = old_flag
 
     def __call__(self, *args, **kwargs):
-        return self.forward(*args, **kwargs)
+        # (See #5078) super().__call__ is used when the method is injected by a
+        # mixin class. To keep backward compatibility, the injected one is
+        # prioritized over forward().
+        forward = getattr(super(Link, self), '__call__', None)
+        if forward is None:
+            forward = self.forward
+        return forward(*args, **kwargs)
 
     def __setattr__(self, name, value):
         if self.within_init_scope and isinstance(value, variable.Parameter):


Test patch:
diff --git a/tests/chainer_tests/test_link.py b/tests/chainer_tests/test_link.py
--- a/tests/chainer_tests/test_link.py
+++ b/tests/chainer_tests/test_link.py
@@ -80,6 +80,22 @@ def test_assign_var_in_init_scope(self):
             self.link.p = p
         self.assertTrue(all(p is not param for param in self.link.params()))
 
+    def test_call_injected_with_mixin(self):
+        call = mock.MagicMock()
+        call.return_value = 3
+
+        class CallMixin(object):
+            __call__ = call
+
+        class InjectedLink(chainer.Link, CallMixin):
+            pass
+
+        link = InjectedLink()
+        ret = link(1, a=2)
+
+        call.assert_called_once_with(1, a=2)
+        assert ret == call.return_value
+
     def test_add_param(self):
         with testing.assert_warns(DeprecationWarning):
             self.link.add_param('z', (2, 3))