Theano/Theano — pull request #894 by goodfeli (2012-08-29 03:57:16+00:00)
Title: Fix theano.gradient.grad

-Updates theano.gradient.grad to correctly discriminate between inputs that are not connected to the cost and inputs that have zero gradient
-Updates theano.gradient.grad to compute only those gradients requested by the user
-Removes the UncomputableOp machinery that was used to hack NaNs into the old grad method
-Introduces NaNType, used to represent undefined or unimplemented gradients
-Adds an optional connection_pattern method to the definition of the op contract. Currently this is only used by the shape op, and will probably only be needed extremely rarely in the future.
-Adds more unit tests to make sure gradient.grad works correctly
-Changes some tests to KnownFailureTest. In one case, the test asserted that a disconnected input error was raised when one shouldn't be. Later this test can be changed to assert that the grad method does the right thing, but we are still arguing on theano-dev about what the right thing is (0 or NaN) in this case. Another test takes a gradient and then asserts that it gets optimized in a specific way, but the new grad method returns an equivalent expression that doesn't get optimized in the same way.

I recommend when reading this pull request you look at the final diff rather than each commit individually. I made several changes in the process of finishing this pull request so several lines in the early commits were later removed.