Theano/Theano — pull request #2248 by carriepl (2014-11-10 16:02:20+00:00)
Title: Scan opt to push out Dot products outside of Scan's inner graph

This is still work in progress. Since dealing with Scan is non-trivial, I just want the opportunity to get some feedback to make sure I'm not going in the wrong direction. At the very least, I still have to do the following :
- Document the code further
- Add tests to validate that the optimization is applied properly

This scan optimization covers the following case :
- One of the inner graph's output is produced by a Dot operation
-The Dot's inputs are a 2D non-sequence input to scan and a 1D tensor produced by scan's inner function
In the covered case, the optimization adds the 1D intermediate variable as a new scan output and performs the Dot outside of Scan with this new output.

On CPU (i7-950) with input sizes (500,) and (500, 500), the following function
a = T.vector()
b = T.matrix()
c = T.dot(a, b)
f = theano.function([a, b], T.jacobian(c, a))
goes from taking 0.06 seconds to 0.03 seconds by execution (averaged over 100 runs)