Repository: Theano/Theano

Issue:
Grad illegally returned an integer-valued variable
Hi!  Wanted to mention that a few tests broke in pymc3 when moving from 0.9.0.dev3 to 0.9.0.dev4

You can see some failures [here](https://travis-ci.org/pymc-devs/pymc3/jobs/172281533), but the gist is 

`TypeError: Elemwise{mul,no_inplace}.grad illegally  returned an integer-valued variable. (Input index 1, dtype bool)`.  

I'm looking at how to fix this in the pymc3 code -- it would be helpful to know if this is considered a bug, and if there's an easy fix!



Discussion:
I don't see the error at the URL.

We recently added a bool dtype in Theano. And we made comparison return
that instead of int8/16.

From the error line you gave, it seem the problem is in the method
Elemwise.grad(). I checked rapidly, and there is some special code for int,
but it seem not being triggered when the inputs is a bool.

I'll try to reproduce it.

On Tue, Nov 1, 2016 at 10:43 PM, Colin notifications@github.com wrote:

> Hi! Wanted to mention that a few tests broke in pymc3 when moving from
> 0.9.0.dev3 to 0.9.0.dev4
> 
> You can see some failures here
> https://travis-ci.org/pymc-devs/pymc3/jobs/172281533, but the gist is
> 
> TypeError: Elemwise{mul,no_inplace}.grad illegally returned an
> integer-valued variable. (Input index 1, dtype bool).
> 
> I'm looking at how to fix this in the pymc3 code -- it would be helpful to
> know if this is considered a bug, and if there's an easy fix!
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> https://github.com/Theano/Theano/issues/5171, or mute the thread
> https://github.com/notifications/unsubscribe-auth/AALC-8O85SJ7YnQc75M_z_Q0JsKKfr4Rks5q5_i9gaJpZM4KmzVF
> .

I'm not able to reproduce it rapidly. But I remember there was some fixes.
@abergeron do you think this is already fixed in Theano or we have
something to do?

On Wed, Nov 2, 2016 at 11:43 AM, Frédéric Bastien <
frederic.bastien@gmail.com> wrote:

> I don't see the error at the URL.
> 
> We recently added a bool dtype in Theano. And we made comparison return
> that instead of int8/16.
> 
> From the error line you gave, it seem the problem is in the method
> Elemwise.grad(). I checked rapidly, and there is some special code for int,
> but it seem not being triggered when the inputs is a bool.
> 
> I'll try to reproduce it.
> 
> On Tue, Nov 1, 2016 at 10:43 PM, Colin notifications@github.com wrote:
> 
> > Hi! Wanted to mention that a few tests broke in pymc3 when moving from
> > 0.9.0.dev3 to 0.9.0.dev4
> > 
> > You can see some failures here
> > https://travis-ci.org/pymc-devs/pymc3/jobs/172281533, but the gist is
> > 
> > TypeError: Elemwise{mul,no_inplace}.grad illegally returned an
> > integer-valued variable. (Input index 1, dtype bool).
> > 
> > I'm looking at how to fix this in the pymc3 code -- it would be helpful
> > to know if this is considered a bug, and if there's an easy fix!
> > 
> > —
> > You are receiving this because you are subscribed to this thread.
> > Reply to this email directly, view it on GitHub
> > https://github.com/Theano/Theano/issues/5171, or mute the thread
> > https://github.com/notifications/unsubscribe-auth/AALC-8O85SJ7YnQc75M_z_Q0JsKKfr4Rks5q5_i9gaJpZM4KmzVF
> > .

I can't be 100% certain that it is fixed, but all of our grad tests don't seem to trigger this.  At this point I'd say have code to reproduce would be necessary.

I'll get an example tonight- thanks for looking!

On Wed, Nov 2, 2016, 11:54 AM abergeron notifications@github.com wrote:

> I can't be 100% certain that it is fixed, but all of our grad tests don't
> seem to trigger this. At this point I'd say have code to reproduce would be
> necessary.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> https://github.com/Theano/Theano/issues/5171#issuecomment-257907937, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ACMHEEW6-0RhwOwjVSzrZB_Ejzrou3Arks5q6LIxgaJpZM4KmzVF
> .

Found a small example on the pymc3 side (https://github.com/pymc-devs/pymc3/issues/1496) -- you _should_ be able to pip install `pymc3`, and install theano master to reproduce.  I'll keep working to get an example that doesn't require pymc3, to at least figure out which library has a bug!


Resolving patch:
diff --git a/theano/tensor/elemwise.py b/theano/tensor/elemwise.py
--- a/theano/tensor/elemwise.py
+++ b/theano/tensor/elemwise.py
@@ -9,6 +9,7 @@
 import theano
 from theano import gof
 from theano.compat import izip
+from theano.configparser import change_flags
 from theano.gof import Apply, Op, OpenMPOp
 from theano import scalar
 from theano.scalar import get_scalar_type
@@ -667,8 +668,8 @@ def grad(self, inputs, ograds):
         # TODO: make sure that zeros are clearly identifiable
         # to the gradient.grad method when the outputs have
         # some integer and some floating point outputs
-        if False in [str(out.type.dtype).find('int') == -1
-                     for out in outs]:
+        if any(out.type.dtype not in theano.tensor.continuous_dtypes
+               for out in outs):
             # For integer output, return value may
             # only be zero or undefined
             # We don't bother with trying to check
@@ -684,7 +685,7 @@ def grad(self, inputs, ograds):
                     new_rval.append(elem)
                 else:
                     elem = ipt.zeros_like()
-                    if str(elem.type.dtype).find('int') != -1:
+                    if str(elem.type.dtype) not in theano.tensor.continuous_dtypes:
                         elem = elem.astype(theano.config.floatX)
                     assert str(elem.type.dtype).find('int') == -1
                     new_rval.append(elem)
@@ -724,12 +725,7 @@ def grad(self, inputs, ograds):
     def _bgrad(self, inputs, ograds):
         # returns grad, with respect to broadcasted versions of inputs
 
-        prev_setting = theano.config.compute_test_value
-
-        try:
-
-            theano.config.compute_test_value = 'off'
-
+        with change_flags(compute_test_value='off'):
             def as_scalar(t):
                 if isinstance(t.type, (NullType, DisconnectedType)):
                     return t
@@ -741,10 +737,6 @@ def as_scalar(t):
             for igrad in scalar_igrads:
                 assert igrad is not None, self.scalar_op
 
-        finally:
-
-            theano.config.compute_test_value = prev_setting
-
         if not isinstance(scalar_igrads, (list, tuple)):
             raise TypeError('%s.grad returned %s instead of list or tuple' %
                             (str(self.scalar_op), str(type(scalar_igrads))))


Test patch:
diff --git a/theano/tensor/tests/test_elemwise.py b/theano/tensor/tests/test_elemwise.py
--- a/theano/tensor/tests/test_elemwise.py
+++ b/theano/tensor/tests/test_elemwise.py
@@ -1154,6 +1154,11 @@ def test_any_grad(self):
 
 
 class TestElemwise(unittest_tools.InferShapeTester):
+    def test_elemwise_grad_bool(self):
+        x = theano.tensor.scalar(dtype='bool')
+        y = theano.tensor.bscalar()
+        z = x * y
+        dx, dy = theano.grad(z, [x, y])
 
     def test_infer_shape(self):