Repository: Theano/Theano

Issue:
ERROR (theano.gof.opt): Optimization failure due to: local_softmax_dnn_grad
I have a cost function that looks like:

``` python
cost = T.nnet.softmax(matrix.flatten()).sum()
```

where `matrix` is the result of some other computation and during optimization I get

``` python
ERROR (theano.gof.opt): Optimization failure due to: local_softmax_dnn_grad
ERROR (theano.gof.opt): node: SoftmaxGrad(TensorConstant{(1, 65536) of 1.0}, HostFromGpu.0)
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
  File "/home/jesse/Development/Theano/theano/gof/opt.py", line 1922, in process_node
    replacements = lopt.transform(node)
  File "/home/jesse/Development/Theano/theano/sandbox/cuda/dnn.py", line 3113, in local_softmax_dnn_grad
    if isinstance(n.owner.op, HostFromGpu):
AttributeError: 'NoneType' object has no attribute 'op'
```

Works fine if I switch to

``` python
cost = T.sqr(matrix.flatten()).sum()
```

or

``` python
matrix_f = matrix.flatten()
cost = (T.nnet.softmax(matrix_f)*matrix_f).sum()
```



Discussion:
Indeed, that line should check that `n.owner` is not None before accessing its `.op` attribute.
That issue does not seem to be there in the new back-end.
Note that the reason it is happening in your case (and not more frequently) is that your cost is constant and 1 all the time, and probably due to shape inference the shapes are considered constant as well, so the input to `SoftmaxGrad` is `TensorConstant{(1, 65536) of 1.0}`, and not the result of another computation.

> your cost is constant and 1

Right, the bug actually helped me find the bug in my code.

This doesn't seem like a very important edge case, but I can send a PR that adds the check.

Thanks, that would be nice!


Resolving patch:
diff --git a/theano/sandbox/cuda/dnn.py b/theano/sandbox/cuda/dnn.py
--- a/theano/sandbox/cuda/dnn.py
+++ b/theano/sandbox/cuda/dnn.py
@@ -3110,8 +3110,9 @@ def local_softmax_dnn_grad(node):
                 return
             ins = []
             for n in node.inputs:
-                if isinstance(n.owner.op, HostFromGpu):
-                    n = n.owner.inputs[0]
+                if n.owner is not None:
+                    if isinstance(n.owner.op, HostFromGpu):
+                        n = n.owner.inputs[0]
                 if n.ndim != 2:
                     return
                 ins.append(n.dimshuffle(0, 'x', 1, 'x'))


Test patch:
diff --git a/theano/sandbox/cuda/tests/test_dnn.py b/theano/sandbox/cuda/tests/test_dnn.py
--- a/theano/sandbox/cuda/tests/test_dnn.py
+++ b/theano/sandbox/cuda/tests/test_dnn.py
@@ -582,6 +582,19 @@ def test_dnn_softmax_grad(self):
         if cuda.dnn.version() != (3000, 3000):
             utt.verify_grad(softmax_op, [x_val2], mode=mode_with_gpu)
 
+    def test_local_softmax_dnn_grad(self):
+        """
+        Check for optimization error when grad of summed
+        softmax is taken over tensor with fixed shape.
+        """
+        x = T.fvector('x')
+        xp = x.reshape((5, 5))
+        y = T.nnet.softmax(xp.flatten()).sum()
+        g = T.grad(y, x)
+        f = theano.function(inputs=[x], outputs=g, mode=self.mode)
+        assert(any(n for n in f.maker.fgraph.toposort() if
+                   isinstance(n.op, dnn.GpuDnnSoftmaxGrad)))
+
     def test_cudnn_softmax_grad_opt(self):
         # Verify that the SoftmaxGrad -> GpuDnnSoftmaxGrad optimization is
         # applied when cudnn is required