aboSamoor/Theano — pull request #5 by viveksck (2013-04-02 14:56:50+00:00)
Title: WIP:Speeding up GpuAdvancedIncSubTensor1 by writing fast Cuda Code 

This commit does the following:
1. It speeds up the implementation of GpuAdvancedIncSubTensor1 by implementing a fast CUDA kernel:
It finds all the rows which needs to be incremented and increments each cell of the row in parallel on GPU.

2. This is a big win. We get a very good speed up as you can see below
3. This assumes that atomicAdd is supported on the GPU

Baseline: Time per call to GpuAdvancedIncSubtensor1 : 7.30e-02s
98.3% 98.3% 146.042s 146.042s 7.30e-02s 2000 2 2

---------------------------------------------------------------------------------------------------------------------------------

With optimization (time per call to GpuAdvancedIncSubTensor1):6.87e-04s
36.2% 36.2% 1.375s 1.375s 6.87e-04s * 2000 2 2

In the next phase of the optimization, I will make a graph optimization which completely makes all GpuAdvancedIncSubtensor1 operations inplace at the cost of a few GpuAlloc.