Repository: Theano/Theano

Issue:
Change grad() method to L_op in ops that need the outputs for gradient
Do as https://github.com/Theano/Theano/pull/5420/commits/0f5a06d7f7bd8480d52d1d2b754bffd7abd14f21 for other Ops that need it.

At least, those ops need to be modified for this:
- [ ] tensor/signal/pool.py
- [x]  Elemwise
- [x] tensor/slinalg.py 
- [ ] All scalar op under scalar/basic.py,
   - [ ] scalar/basic_scipy.py
   - [ ] theano/tensor/nnet/nnet.py
   - [ ] theano/tensor/nnet/sigm.py
   - [ ] theano/tensor/xlogx.py
- [ ] tensor/compile/builders.py OpFromGraph (started in https://github.com/Theano/Theano/pull/5641)
Is there other ops? Should we look at all ops in Theano?


Discussion:
Hi @nouiz , I would like to work on this issue. Do I need to make a similar change as the one in 0f5a06d , in the ops listed below?
That is exact. Start at the top of the file. They are easier to do.

On Thu, Jan 19, 2017 at 6:10 AM, wazeerzulfikar <notifications@github.com>
wrote:

> Hi @nouiz <https://github.com/nouiz> , I would like to work on this
> issue. Do I need to make a similar change as the one in 0f5a06d
> <https://github.com/Theano/Theano/commit/0f5a06d7f7bd8480d52d1d2b754bffd7abd14f21>
> , in the ops listed below?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/Theano/Theano/issues/5426#issuecomment-273747879>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AALC-6I_Z_uROOxWPsREuhjHIgRnuCtYks5rT0S1gaJpZM4LnAzY>
> .
>

@nouiz , Should all the grad() functions be changed to L_op in these ops, or only the ones which are similar to the one in scan_op.py?

Resolving patch:
diff --git a/theano/tensor/elemwise.py b/theano/tensor/elemwise.py
--- a/theano/tensor/elemwise.py
+++ b/theano/tensor/elemwise.py
@@ -634,11 +634,7 @@ def connection_pattern(self, node):
 
         return [[True for output in node.outputs] for ipt in node.inputs]
 
-    def grad(self, inputs, ograds):
-
-        outs = self(*inputs)
-        if not isinstance(outs, (list, tuple)):
-            outs = [outs]
+    def L_op(self, inputs, outs, ograds):
 
         # compute grad with respect to broadcasted input
         rval = self._bgrad(inputs, ograds)
@@ -1927,12 +1923,10 @@ def __str__(self):
             str(self.acc_dtype)
         )
 
-    def grad(self, inp, grads):
+    def L_op(self, inp, out, grads):
         x, = inp
 
-        out = self(*inp)
-
-        if out.dtype not in theano.tensor.continuous_dtypes:
+        if out[0].dtype not in theano.tensor.continuous_dtypes:
             return [x.zeros_like(dtype=theano.config.floatX)]
 
         gz, = grads
@@ -1985,7 +1979,7 @@ def __setstate__(self, dct):
         if 'no_zeros_in_input' not in dct:
             self.no_zeros_in_input = False
 
-    def grad(self, inp, grads):
+    def L_op(self, inp, out, grads):
         """
         The grad of this Op could be very easy, if it is was not for the case
         where zeros are present in a given "group" (ie. elements reduced
@@ -2034,9 +2028,7 @@ def grad(self, inp, grads):
         prod_in, = inp
         gz, = grads
 
-        out = self(*inp)
-
-        if (out.dtype in theano.tensor.discrete_dtypes or
+        if (out[0].dtype in theano.tensor.discrete_dtypes or
                 self.acc_dtype in theano.tensor.discrete_dtypes):
             # There is an int conversion in the way
             return [prod_in.zeros_like(dtype=theano.config.floatX)]


Test patch: