Theano/Theano — pull request #132 by ynd (2011-10-20 19:10:02+00:00)
Title: added sparse dot

*new pull request with the requested changes*

This adds the sparse dot. The difference with the structured_dot are:
1. Support for sparse operands anywhere(Sparse x Dense, Dense x Sparse, Spare x Sparse) whereas structured_dot supports only supports (Sparse x Dense, with DxS supported using transposes).
2. Structured_dot has a special semantic and is not the true dot because it doesn't calculate the gradient wrt to zero elements. Of course in some cases this doesn't matter, but it does sometimes (second layer with sparse hiddens, or sparse weights for ex).
3. Structured_dot will not yield speed-ups when dealing with sparse inputs. The reason is that it doesn't perform the weight update inplace which is very important. I have an Op called Usmm for that. Usmm is the equivalent of Gemm in sparse blas.