Repository: chainer/chainer

Issue:
`F.forget` does not work correctly when inputs are bare ndarray
I found that `F.forget` does not work correctly when inputs to `F.forget` are bare `xp.ndarray` (rather than `chainer.Variable`). When all inputs to `F.forget` are bare ndarray, `backward` of `F.forget` is completely ignored and it is not called, even if trainable parameters are actually involved in the "forgotten" process (i.e., `func` parameter of `F.forget`).

As we usually want to use `F.forget` at the beginning of DNNs, it is natural to give bare ndarrays to `F.forget`. Therefore, I think this is a critical problem.

My minimal reproduction code is as follows. I used Chainer 2.1.0.

```python
import chainer
import chainer.links as L
import chainer.functions as F
import numpy as np

lnk1 = L.Scale(axis=0, W_shape=(1,))
lnk2 = L.Scale(axis=0, W_shape=(1,))

def forward(x, use_forget):
    if use_forget:
        x = F.forget(lambda x_: lnk1(x_), x)
    else:
        x = lnk1(x)

    return lnk2(x)

def print_grad(x, use_forget):
    lnk1.cleargrads()
    y = forward(x, use_forget=use_forget)
    y.backward()
    print(lnk1.W.grad)

in_variable = chainer.Variable(np.array([1], dtype=np.float32))
in_ndarray = np.array([1], dtype=np.float32)

#
# Four test cases
#
print('Chainer version:', chainer.__version__)
print_grad(in_variable, use_forget=False)  # OK
print_grad(in_ndarray, use_forget=False)   # OK
print_grad(in_variable, use_forget=True)   # OK
print_grad(in_ndarray, use_forget=True)    # NG
```

```sh
$ python forget_test.py
Chainer version: 2.1.0
[ 1.]
[ 1.]
[ 1.]
None
```


Discussion:
Thank you for reporting this.

This behavior is a result of calling [as_variable()](https://github.com/chainer/chainer/blob/master/chainer/variable.py#L1267-L1294) inside the `FunctionNode` to wrap all inputs in `Variable`s. This may unchain the computational graph as ndarrays that are wrapped in `Variables` using this function will have their `requires_grad` attribute set to `False`. A function only receiving inputs of this flag configuration will unchain the graph. In this case, `F.forget` is cut from the graph.

I think fixing this issue might require non-trivial changes. @niboshi 
How about setting `requires_grad=True` on such variables?

Resolving patch:
diff --git a/chainer/functions/util/forget.py b/chainer/functions/util/forget.py
--- a/chainer/functions/util/forget.py
+++ b/chainer/functions/util/forget.py
@@ -101,10 +101,18 @@ def forget(func, *xs):
 
     .. note::
 
-      ``F.forget`` does not support functions which behave differently in
-      multiple calls with the same inputs, such as
-      :meth:`F.dropout() <chainer.functions.dropout>` and
-      :meth:`F.negative_sampling() <chainer.functions.negative_sampling>`.
+        ``F.forget`` does not support functions which behave differently in
+        multiple calls with the same inputs, such as
+        :meth:`F.dropout() <chainer.functions.dropout>` and
+        :meth:`F.negative_sampling() <chainer.functions.negative_sampling>`.
+
+    .. note::
+
+        In case input argument variables are of class :class:`numpy.ndarray` or
+        :class:`cupy.ndarray` objects, arguments will automatically be
+        converted to :class:`~chainer.Variable`\ s. This conversion takes place
+        to ensure that this function is included in the computational graph to
+        enable backward computations.
 
     Args:
         func (callable): A function to call. It needs to be called with
@@ -118,4 +126,6 @@ def forget(func, *xs):
         the method returns a tuple too.
 
     """
+    xs = tuple(x if isinstance(x, variable.Variable) else
+               variable.Variable(x, requires_grad=True) for x in xs)
     return Forget(func)(*xs)


Test patch: