Repository: Theano/Theano

Issue:
Theano TypeError message far from useful
Currently I have encountered a TypeError in using T.grad to compute a gradient regarding outputs of a T.scan.
Most debugging methods provided by Theano involves with compiling the function, so I'm really out of choice right now. The error is:
TypeError: Cannot convert Type TensorType(float64, 3D) (of Variable Subtensor{:int64:}.0) into Type TensorType(float64, (False, True, False)). You can try to manually convert Subtensor{:int64:}.0 into a TensorType(float64, (False, True, False)).

Let's be serious, this kind of error message is utterly junk.  The suggestion _You can try to manually convert Subtensor{:int64:}.0 into a TensorType(float64, (False, True, False))_ is obviously useless, who knows where did that Subtensor comes from? 
It's not hard to catch this TypeError in theano calling stack(say, catch it in scan-op) and provide more useful messages. 
Theano should provide more information in exception handling.


Discussion:
Could you share your code that led to this problem? We can then check how the error message in this case could be improved (If required) 
Sorry for the delay. I was using theano to implement a single-layer RNN and the error appeared in computing gradients.
The code is below:
```python
from theano import tensor as T
import theano
import numpy as np

#a single layer RNN

def cross_entro(outputTensor, targetTensor):
    output = T.flatten(outputTensor, outdim=2).dimshuffle(1, 0) #exchange dimension
    dot_product =  -T.sum(T.dot(targetTensor, T.log(output)), axis=1)
    return T.mean(dot_product)

numpy_rng = np.random.RandomState(1234)

numHUnits = 100
mini_batch_size = 1

'''
1th dimension: batch size
2rd dimension: time step size
'''
inputShape = (-1, -1, 1)#the input shape to the rnn

targetTensor = T.tensor3()
inputTensor = T.tensor3()

#dumy initialization
V= theano.shared( numpy_rng.uniform(low=0, high=1, size=(numHUnits, inputShape[-1])) )
W= theano.shared( numpy_rng.uniform(low=0, high=1, size=(numHUnits, numHUnits)) )
U= theano.shared( numpy_rng.uniform(low=0, high=1, size=(inputShape[-1], numHUnits)) )

input = inputTensor.dimshuffle(1, 0, 2)

def forward_prop_step(x_t, s_t_prev, U, V, W):
    s_t = T.nnet.softmax(T.dot(x_t, U) + T.dot(s_t_prev, W))
    o_t = T.nnet.softmax(T.dot(s_t, V))
    return o_t, s_t

#iterate through time steps
[o_t, s_t], updates = theano.scan(fn=forward_prop_step,
                                    sequences=input,
                                    outputs_info=[None, dict(initial=np.zeros((mini_batch_size, numHUnits)))], #make s_t_prev broadcastba
                                    non_sequences=[U, V, W])
#switch batch number with seq number
outputTensor = o_t.dimshuffle(1, 0, 2)
cost = cross_entro(outputTensor, targetTensor)

#TODO this is where exception is thrown
gs = T.grad(cost, [V, W, U]) #compute gradients
```
The error message I got is:
```
TypeError: Cannot convert Type TensorType(float64, 3D) (of Variable Subtensor{:int64:}.0) into Type TensorType(float64, (False, True, False)). You can try to manually convert Subtensor{:int64:}.0 into a TensorType(float64, (False, True, False)).
```
The solution of your problem is hidden in the comment `make s_t_prev broadcastba` in your code:
```python
#iterate through time steps
[o_t, s_t], updates = theano.scan(fn=forward_prop_step,
                                    sequences=input,
                                    outputs_info=[None, dict(initial=np.zeros((mini_batch_size, numHUnits)))], #make s_t_prev broadcastba
                                    non_sequences=[U, V, W])
```
Because `mini_batch_size = 1`, the initial tensor will get broadcastable pattern `(True, False)`, which doesn't match the broadcastable pattern of the result of the `forward_prop_step`.

Unbroadcasting the inititial value solves the problem on my system:
```python
#iterate through time steps
initial_s_t = np.zeros((mini_batch_size, numHUnits))
initial_s_t_var = T.unbroadcast(T.constant(initial_s_t), 0, 1)
[o_t, s_t], updates = theano.scan(fn=forward_prop_step,
                                    sequences=input,
                                    outputs_info=[None, dict(initial=initial_s_t_var)],
                                    non_sequences=[U, V, W])
```
I don't know if that's something that the scan op could do automatically.
I think this issue is the same as: https://github.com/Theano/Theano/issues/5224

The advantage is that the code to reproduce it is in Theano only, so this is a better starting point for a test.

Hopefully, we should start to work on this this month.

I don't think scan can do that itself, but the error message can be better. Why I think scan can't do the upcast itself? You explicitly told scan that the output of each iteration will have the broadcast pattern (True, False). It is the same as telling the dtype will be float64. Theano can't silently change that to (False, False) (aka float32).

The other option is to make Theano infer that the output is (True, False). But there is always cases(at least in theory) that this won't be possible. So I think we should:

1) Make a better error message that give the stack trace when the initial value is provided. Where we tell exactly which scan input/output is causing problem. Detect if the problem is only with the broadcast pattern and not the dtype, talk about T.patternbroadcast() as a way to fix it.
2) Investigate if in this particular case we could infer the (True, False) pattern automatically.

Now I'm not so sure about the fix.

Mostly, a broadcast pattern of True mean the shape will be 1. But a broadcast pattern of False can mean we aren't able to infer that it will be 1, but can also mean the shape won't be 1. As we currently can't make this distinction, it seem reasonable to don't raise that error as we aren't sure the problem is in the user graph or our inability to infer the shape.

So another good fix would be to acccept change of broadcast pattern from True to False at that place, but make sure that at run time we raise an error is the shape don't match.

Resolving patch:
diff --git a/theano/scan_module/scan_op.py b/theano/scan_module/scan_op.py
--- a/theano/scan_module/scan_op.py
+++ b/theano/scan_module/scan_op.py
@@ -402,6 +402,36 @@ def make_node(self, *inputs):
                     'by using dimshuffle or shape_padleft. '
                     )
 
+        def check_broadcast(v1, v2):
+            """ Checks that the broadcast pattern of v1 and v2.
+
+            Controls that the broadcast pattern of the variable provided as
+            input to `scan` matches the broadcast pattern provided in
+            `output_info`. It raises an error when they don't match. The
+            typical case is when the user provides either the input or the
+            `output_info` (but not both) with a dimension fixed to 1,
+            which may wrongly be interpreted as broadcastable.
+
+            """
+            if (not hasattr(v1, 'broadcastable') and
+                    not hasattr(v2, 'broadcastable')):
+                return
+            msg = ("The broadcast pattern of the output of scan (%s) is "
+                   "inconsistent with the one provided in `output_info` "
+                   "(%s). The output on axis %d is `%r`, but it is `%r` on "
+                   "axis %d in `output_info`. This can happen if one of the "
+                   "dimension is fixed to 1 in the input, while it is still "
+                   "variable in the output, or vice-verca. You have to make "
+                   "them consistent, e.g. using theano.tensor."
+                   "{patternbroadcast,unbroadcast,addbroadcast}.")
+            size = min(len(v1.broadcastable), len(v2.broadcastable))
+            for n, (b1, b2) in enumerate(zip(v1.broadcastable[-size:],
+                                             v2.broadcastable[-size:])):
+                if b1 != b2:
+                    a1 = n + size - len(v1.broadcastable) + 1
+                    a2 = n + size - len(v2.broadcastable) + 1
+                    raise TypeError(msg % (v1.type, v2.type, a1, b1, b2, a2))
+
         def format(var, as_var):
             """
             This functions ensures that ``out`` has the same dtype as
@@ -429,6 +459,7 @@ def format(var, as_var):
         argoffset = 0
         for inner_seq, outer_seq in zip(self.inner_seqs(self.inputs),
                                         self.outer_seqs(inputs)):
+            check_broadcast(outer_seq, inner_seq)
             new_inputs.append(format(outer_seq, as_var=inner_seq))
 
         argoffset += len(self.outer_seqs(inputs))


Test patch:
diff --git a/theano/scan_module/tests/test_scan.py b/theano/scan_module/tests/test_scan.py
--- a/theano/scan_module/tests/test_scan.py
+++ b/theano/scan_module/tests/test_scan.py
@@ -5495,3 +5495,23 @@ def _active(x, pre_h):
     updates = [(W_x, W_x - 0.1 * gW_x)]
     f = theano.function([X], outputs=cost, updates=updates)
     f(numpy.random.rand(10, in_size).astype(X.dtype))
+
+
+class TestInconsistentBroadcast(unittest.TestCase):
+
+    def test_raise_error(self):
+        x = tensor.tensor3()
+        initial_x = tensor.constant(numpy.zeros((1, 10)))
+        y, updates = theano.scan(fn=lambda x, prev_x: x + prev_x,
+                                 sequences=x,
+                                 outputs_info=[dict(initial=initial_x)])
+        # Error, because the broadcast patterns are inconsistent.
+        with self.assertRaises(TypeError):
+            gs = tensor.grad(y.sum(), x)
+
+        # No error here, because the broadcast patterns are consistent.
+        initial_x = tensor.unbroadcast(initial_x, 0, 1)
+        y, updates = theano.scan(fn=lambda x, prev_x: x + prev_x,
+                                 sequences=x,
+                                 outputs_info=[dict(initial=initial_x)])
+        gs = tensor.grad(y.sum(), x)