Repository: Theano/Theano

Issue:
GpuAdvancedSubtensor of tensor3/4 gives incorrect output with new gpuarray backend.
I've found an inconsistency in the output of advanced indexing on 3D/4D tensors with the gpuarray backend (I wasn't able to reproduce the issue with a matrix). I wasn't sure whether to submit this here or on libgpuarray so please let me know if it's in the wrong place.

Here's a small example that produces the problem:

```python
import numpy as np
import theano.tensor as tt

X = tt.tensor3("X")

xx = np.zeros((3,2,2), np.float32)
for i in range(3):
    for j in range(2):
        for k in range(2):
            xx[i,j,k] = 100*i+10*j+k

b_idx = np.zeros((2,2), np.int32)
b_idx[0,1] = 1
b_idx[1,1] = 2

r_idx = np.arange(xx.shape[1])[np.newaxis, :]
c_idx = np.arange(xx.shape[2])[np.newaxis, :]

out = X[b_idx, r_idx, c_idx].eval({X:xx})
```

If you run the code with `THEANO_FLAGS="mode=DebugMode,device=cuda"`, it raises a BadOptimizationError when the AdvancedSubtensor op gets moved to the GPU. (There's no error running w/device=gpu)

```
theano.compile.debugmode.BadOptimization: BadOptimization Error 
  Variable: id 140699957263104 HostFromGpu(gpuarray).0
  Op HostFromGpu(gpuarray)(GpuAdvancedSubtensor.0)
  Value Type: <class 'numpy.ndarray'>
  Old Value shape, dtype, strides: (2, 2) float32 (8, 4)

  Old Value:  [[   0.  111.]
               [   0.  211.]]
  New Value shape, dtype, strides: (2, 2) float32 (8, 4)

  New Value:  [[   0.  101.]
               [   0.  111.]]
  Max Abs Diff:  100.0
  Mean Abs Diff:  27.5
  Median Abs Diff:  5.0
  Std Abs Diff:  42.0565
  Value at Max Diff:  (111.0, 211.0)
  Max Rel Diff:  0.310559
  Mean Rel Diff:  0.0894322
  Median Rel Diff:  0.0235849
  Std Rel Diff:  0.129112
  Value at Max Diff:  (111.0, 211.0)

  Reason:  GraphToGPU
  Old Graph:
  AdvancedSubtensor [id A] ''   
   |X [id B]
   |TensorConstant{[[0 1]
 [0 2]]} [id C]
   |TensorConstant{[[0 1]]} [id D]
   |TensorConstant{[[0 1]]} [id D]

  New Graph:
  HostFromGpu(gpuarray) [id E] ''   
   |GpuAdvancedSubtensor [id F] ''   
     |GpuFromHost<None> [id G] ''   
     | |X [id B]
     |TensorConstant{[[0 1]
 [0 2]]} [id C]
     |TensorConstant{[[0 1]]} [id D]
     |TensorConstant{[[0 1]]} [id D]
```

I'm running the latest master versions of theano and libgpuarray:

```
>>> theano.__version__
'0.9.0rc3.dev-ee9eaeb04fa3be723341d0c7ab9918391d0b89c8'
```

```
libgpuarray]$ git rev-parse HEAD
23c8f004968f2d7703612f2ebb0e2da1b112bc7d
```


Discussion:
Confirmed, I'm working on a fix.

Resolving patch:
diff --git a/theano/gpuarray/subtensor.py b/theano/gpuarray/subtensor.py
--- a/theano/gpuarray/subtensor.py
+++ b/theano/gpuarray/subtensor.py
@@ -552,7 +552,7 @@ def perform(self, node, inputs, out_):
         # build the strides
         strides = [1]
         for i in range(p - 1, 0, -1):
-            stride = x.shape[i] * strides[-1]
+            stride = x.shape[i] * strides[0]
             strides.insert(0, stride)
 
         # build the indices and use it


Test patch:
diff --git a/theano/tensor/tests/test_subtensor.py b/theano/tensor/tests/test_subtensor.py
--- a/theano/tensor/tests/test_subtensor.py
+++ b/theano/tensor/tests/test_subtensor.py
@@ -1603,6 +1603,26 @@ def test_advanced_indexing(self):
         cmd = f2(0, 1, 2) == aa[[0, 1, 2], :, 0:2]
         self.assertTrue(cmd.all())
 
+    def test_adv_sub_3d(self):
+        # Reported in https://github.com/Theano/Theano/issues/5674
+        X = tensor.tensor3("X")
+
+        xx = numpy.zeros((3, 2, 2), config.floatX)
+        for i in range(3):
+            for j in range(2):
+                for k in range(2):
+                    xx[i, j, k] = 100 * i + 10 * j + k
+
+        b_idx = numpy.zeros((2, 2), 'int32')
+        b_idx[0, 1] = 1
+        b_idx[1, 1] = 2
+
+        r_idx = numpy.arange(xx.shape[1])[numpy.newaxis, :]
+        c_idx = numpy.arange(xx.shape[2])[numpy.newaxis, :]
+
+        out = X[b_idx, r_idx, c_idx].eval({X: xx})
+        utt.assert_allclose(out, xx[b_idx, r_idx, c_idx])
+
     def test_grad(self):
         ones = numpy.ones((1, 3), dtype=self.dtype)
         n = self.shared(ones * 5, broadcastable=(True, False))