You want it to be absolutly on the GPU, so don't use theano.shared that don't give you this assection. use theano.sandbox.cuda.float32_shared_constructor()