jaberg/Theano — comment on commit a019900862d2e76ccb4e868634087e325fe594e0
File:  line 0
By nouiz:

The more I look into it, the uglier the current state is(and not well documented)... There is multiple phase that insert inplace and 2 phases that insert gpu op. I remember why we need the 2 gpu insertion phase, and only why we need 2 of the multiple insert phase. We need to the the blas inplace first, then the other one after as this will give the best speed up. Do you remember why we need to have the *subtensor* inplace before other inplace opt? Here is what I found:

('gpu_opt', 48.5), ('add_destroy_handler', 49.5), ('inplace_setsubtensor', 60), ('local_inplace_incsubtensor1', 60), ('InplaceBlasOpt', 70.0), ('InplaceGpuBlasOpt', 70.0), ('make_scipy_blas_destructive', 70.0), ('elemwise_fusion', 71.0), ('gpu_elemwise_fusion', 71.0), ('gpu_after_fusion', 71.1), ('gpu_inplace_elemwise_opt', 75), ('gpu_scanOp_make_inplace', 75), ('inplace_opt', 75), ('scanOp_make_inplace', 75), ('random_make_inplace', 99), ('CURAND_destructive', 99)

So there is already this step that exist: InplaceGpuBlasOpt at step 70.0 that is after the gpu opt. So I would put them the new gpu blas opt in that phase.

this PR should not try to fix all the opt problem. I don't want to do that just before a release. I'll make a ticket to duplicate that step after the second gpu transfer phase. As this second phase could move other blas op on the gpu. But this will happen only when we have elemwise fusion that take other dtype then float32 inside the fused elemwise. In fact, as this will disapear with the new gpu nd array, I won't make a tiket to fix this problem.

So in conclusion, put them in InplaceGpuBlasOpt.