Repository: Theano/Theano

Issue:
ERROR (theano.gof.opt): Optimization failure due to: local_inplace_sparseblockouter
I installed the cudnn 5.0 and upgrade theano to the newest develop version on linux 14.04 LTS. 
When I run .py filtes under ../theano/tests/ folder, for example using the command `python test_gradient.py`, the errors come out as follows:

`dl@lab401-Z97X:~/gds/dl_software/Theano/theano/tests$ python test_gradient.py`
`Using gpu device 0: GeForce GTX 980 (CNMeM is disabled, cuDNN None)`
`ERROR (theano.gof.opt): Optimization failure due to: local_inplace_sparseblockgemv`
`ERROR (theano.gof.opt): node: GpuCAReduce{add}{1}(<CudaNdarrayType(float32, vector)>)`
`ERROR (theano.gof.opt): TRACEBACK:`
`ERROR (theano.gof.opt): Traceback (most recent call last):`
`File "/home/dl/gds/dl_software/Theano/theano/gof/opt.py", line 1779, in process_node`
`replacements = lopt.transform(node)`
`File "/home/dl/gds/dl_software/Theano/theano/gpuarray/opt.py", line 904, in` `local_inplace_sparseblockgemv`
`if isinstance(node.op, GpuSparseBlockGemv) and not node.op.inplace:`
`TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types`

Did anyone encounter the same error before?
I have installed my cudnn correctly following [http://deeplearning.net/software/theano/library/sandbox/cuda/dnn.html](url).
By the way, caffe could run cudnn based layer normally.



Discussion:
@nouiz This issue was introduced by the latest commit (0e312ca394ca1559aa4abc50ee247aa444cc3d19), also affects cuDNN 4.
Until it is resolved, check out 2ec0fa504a7c0d231601788a5481b7687a7303ba, there it works fine.

I'm not able to reproduce that error with 5c02b98ed05b9706b1ffe672a32bd69fb2ce5b50, can you check if it has been fixed in the new master?

I confirm that in 5c02b98 this works after patching the nvcc_compiler.py with the workaround mentioned in #4425, but it does not anymore in the latest e554dd5.

I got this problem today too. Do you use the new gpu back-end or the
current one? The problem is related to the order of import. If you only use
the current back-end, delete your Theano cache, this will fix it for now.

The problem appear at least when you use device=cuda*, at import this
trigger the compilation of some Theano function to test the cuda driver in
the old back-end. This in turn trigger the load of the cache and this
enable some gpu opt. Those opt try to execute, but the import aren't
finished, so GpuSparseBlockGemv end up being None and this cause problems.

On Thu, May 19, 2016 at 2:00 PM, Philipp Kainz notifications@github.com
wrote:

> I confirm that in 5c02b98
> https://github.com/Theano/Theano/commit/5c02b98ed05b9706b1ffe672a32bd69fb2ce5b50
> this works after patching the nvcc_compiler.py with the workaround
> mentioned in #4425 https://github.com/Theano/Theano/issues/4425, but it
> does not anymore in the latest e554dd5
> https://github.com/Theano/Theano/commit/e554dd5aa0411451a6b011daf07f366d4ceffab6
> .
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub
> https://github.com/Theano/Theano/issues/4531#issuecomment-220404108

I think I have a fix, I'm checking it.


Resolving patch:
diff --git a/theano/sandbox/cuda/opt.py b/theano/sandbox/cuda/opt.py
--- a/theano/sandbox/cuda/opt.py
+++ b/theano/sandbox/cuda/opt.py
@@ -524,7 +524,8 @@ def local_assert_no_cpu_op(node):
 assert_no_cpu_op = theano.tensor.opt.in2out(local_assert_no_cpu_op,
                                             name='assert_no_cpu_op')
 # 49.2 is after device specialization & fusion optimizations for last transfers
-optdb.register('assert_no_cpu_op', assert_no_cpu_op, 49.2)
+optdb.register('gpu_assert_no_cpu_op', assert_no_cpu_op, 49.2,
+               'assert_no_cpu_op')
 
 
 @register_opt()


Test patch: