Theano/Theano — pull request #910 by goodfeli (2012-09-06 19:18:57+00:00)
Title: Consistent & correct handling of integers and gradients

This pull request introduces:
-Documentation of a consistent way of handling gradients and integers
-Type checks that ensure the gradient is always floating point and not an integer
-Type checks that ensure the gradient of an integer is always undefined or 0
-A new unit test checking that the gradient of an autoencoder hidden layer with respect to its input doesn't change if you make the input integer-typed
-An upgraded version of connection_pattern that provides theano with enough information to answer questions like "is variable x a function of variable y?" accurately
-Corrections to all tested ops making sure that their gradients handle integers correctly and consistently
-Type checks preventing the grad method from returning None, which in the past could have meant zero, disconnected, or undefined

Some tests are changed:
-Some tests used to enforce incorrect behavior, such as that the gradient with respect to an integer must be 0. These tests have been corrected.
-One KnownFailureTest said that the correct behavior for the test was under debate. The test now enforces the behavior I document in this pull request.
-This pull request exposes a bug in Rop where Rop does not detect an undifferentiable input. One of the Rop tests is now marked as KnownFailure
-One test using the deprecated compile.module functionality is marked as KnownFailure, because getting the test to pass would be labor intensive and I'd prefer to have someone familiar with compile.module port the test to not use deprecated functionality anymore. If it can be translated to use theano.function / theano.shared it should be easy to make it pass.

When reviewing this pull request, I recommend reading the global diff, rather than individual comments. After submitting the pull request I will go through the global diff and add line notes explaining what each section does.

Please hold all comments about PEP8 stuff until all debate about functionality is done with. I'll clean up the PEP8 as a final post-processing step.

Finally, please remember that the purpose of code review is to improve the correctness, readability, efficiency, etc. of the pull request, not to try to veto any pull request that doesn't implement one of your grandiose dreams. Please keep your code review comments focused on whether the pull request leaves the library in a better state than before the pull request.