aboSamoor/Theano — pull request #1 by viveksck (2013-01-23 22:05:38+00:00)
Title: Theano bug fix: Patch for gradient of AdvancedSubIncTensor1 with code review comments attached: WIP

This addresses the comments from the previous review:
A few Things:
1. Sparse Unit tests pass
(theano-vivek)vvkulkarni@descartes:~/Theano/Theano$ theano-nose ./theano/sparse/tests/
.........../usr/lib/python2.7/dist-packages/scipy/sparse/data.py:54: ComplexWarning: Casting complex values to real discards the imaginary part
return self._with_data(self.data.astype(t))
/home/vvkulkarni/Theano/Theano/theano/sparse/tests/test_basic.py:2021: ComplexWarning: Casting complex values to real discards the imaginary part
expected = data.toarray().astype(o_dtype)
...../usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py:486: SparseEfficiencyWarning: changing the sparsity structure of a csc_matrix is expensive. lil_matrix is more efficient.
SparseEfficiencyWarning)
/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py:486: SparseEfficiencyWarning: changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
SparseEfficiencyWarning)
...........S.........................................S.......S.............................................................................................
----------------------------------------------------------------------
Ran 171 tests in 87.401s

OK (SKIP=3)

2. 2 Unit tests for Tensor fail as they expect a tensor but we are returning a sparse matrix:
======================================================================
ERROR: test_grad_list (theano.tensor.tests.test_basic.T_subtensor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vvkulkarni/Theano/Theano/theano/tensor/tests/test_basic.py", line 3264, in test_grad_list
self.grad_list_(idxs, data)
File "/home/vvkulkarni/Theano/Theano/theano/tensor/tests/test_basic.py", line 3195, in grad_list_
gn = grad(sum(exp(t)), n)
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 529, in grad
grad_dict, wrt, cost_name)
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 1068, in _populate_grad_dict
rval = [access_grad_cache(elem) for elem in wrt]
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 1047, in access_grad_cache
str(node.op), term.ndim, var.ndim))
ValueError: .grad returned a term with 2 dimensions, but 1 are required.

======================================================================
ERROR: test_alloc_constant_folding (theano.tensor.tests.test_basic.TestAlloc)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vvkulkarni/Theano/Theano/theano/tensor/tests/test_basic.py", line 1766, in test_alloc_constant_folding
grad_derp = theano.grad(derp, some_vector)
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 529, in grad
grad_dict, wrt, cost_name)
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 1068, in _populate_grad_dict
rval = [access_grad_cache(elem) for elem in wrt]
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 1028, in access_grad_cache
term = access_term_cache(node)[idx]
File "/home/vvkulkarni/Theano/Theano/theano/gradient.py", line 896, in access_term_cache
input_grads = node.op.grad(inputs, new_output_grads)
File "/home/vvkulkarni/Theano/Theano/theano/tensor/basic.py", line 5844, in grad
return [reshape(g_out, shape(x), ndim=x.ndim),
File "/home/vvkulkarni/Theano/Theano/theano/tensor/basic.py", line 5945, in reshape
rval = op(x, newshape)
File "/home/vvkulkarni/Theano/Theano/theano/gof/op.py", line 381, in __call__
node = self.make_node(*inputs, **kwargs)
File "/home/vvkulkarni/Theano/Theano/theano/tensor/basic.py", line 5799, in make_node
x = as_tensor_variable(x)
File "/home/vvkulkarni/Theano/Theano/theano/tensor/basic.py", line 161, in as_tensor_variable
"Variable type field must be a TensorType.", x, x.type)
TypeError: ('Variable type field must be a TensorType.', SparseVariable{csc,float64}, Sparse[float64, csc])

----------------------------------------------------------------------
Ran 1250 tests in 235.674s

FAILED (KNOWNFAIL=10, SKIP=7, errors=2)

3. There is also no unit test for verifying the gradient if AddSD operation
4. The script run with these changes::
(theano-vivek)vvkulkarni@descartes:~/Downloads/graphs_debug_prints_script$ python rami2.py today.png
size 10000
advanced
The output file is available at today.png
Sum [@A] '' 39
|Gemm{inplace} [@B] '' 36
|_dot22 [@C] '' 33
| |Reshape{2} [@D] '' 31
| | |Flatten{1} [@E] '' 17
| | | | [@F] '' 13
| | | |W [@G]
| | | |Flatten{1} [@H] '' 9
| | | |wrong [@I]
| | |MakeVector [@J] '' 29
| | |Shape_i{0} [@K] '' 7
| | | |wrong [@I]
| | |Elemwise{IntDiv{output_types_preference=transfer_type{0}}}[(0, 0)] [@L] '' 27
| | |Prod [@M] '' 25
| | | |MakeVector [@N] '' 23
| | | |Prod [@O] '' 21
| | | |MakeVector [@P] '' 19
| | | |Prod [@Q] '' 16
| | | | |MakeVector [@R] '' 12
| | | | |Shape_i{0} [@K] '' 7
| | | | |Shape_i{1} [@S] '' 8
| | | | |wrong [@I]
| | | |Shape_i{1} [@T] '' 1
| | | |W [@G]
| | |Shape_i{0} [@K] '' 7
| |H [@U]
|TensorConstant{-1.0} [@V]
|Reshape{2} [@W] '' 30
| |Flatten{1} [@X] '' 14
| | | [@Y] '' 10
| | |W [@G]
| | |Flatten{1} [@Z] '' 2
| | |input [@BA]
| |MakeVector [@BB] '' 28
| |Shape_i{0} [@BC] '' 4
| | |input [@BA]
| |Elemwise{IntDiv{output_types_preference=transfer_type{0}}}[(0, 0)] [@BD] '' 26
| |Prod [@BE] '' 24
| | |MakeVector [@BF] '' 22
| | |Prod [@BG] '' 20
| | |MakeVector [@BH] '' 18
| | |Prod [@BI] '' 15
| | | |MakeVector [@BJ] '' 11
| | | |Shape_i{0} [@BC] '' 4
| | | |Shape_i{1} [@BK] '' 3
| | | |input [@BA]
| | |Shape_i{1} [@T] '' 1
| |Shape_i{0} [@BC] '' 4
|H [@U]
|TensorConstant{1.0} [@BL]
AddSD{inplace} [@BM] '' 50
|CSMProperties{None}.0 [@BN] '' 49
| |CSM [@BO] '' 48
| |CSMProperties{None}.0 [@BP] '' 47
| | |SparseNeg [@BQ] '' 46
| | |MulSD [@BR] '' 45
| | |AddSS [@BS] '' 44
| | | |ConstructSparse [@BT] '' 43
| | | | |W [@G]
| | | | |Reshape{2} [@BU] '' 41
| | | | | |_dot22 [@BV] '' 38
| | | | | | |Alloc [@BW] '' 35
| | | | | | | |TensorConstant{(1, 1) of 1.0} [@BX]
| | | | | | | |Shape_i{0} [@BY] '' 32
| | | | | | | | |Reshape{2} [@D] '' 31
| | | | | | | |Shape_i{1} [@BZ] '' 6
| | | | | | | |H [@U]
| | | | | | |InplaceDimShuffle{1,0} [@CA] 'H.T' 5
| | | | | | |H [@U]
| | | | | |MakeVector [@P] '' 19
| | | | |Flatten{1} [@H] '' 9
| | | |ConstructSparse [@CB] '' 42
| | | |W [@G]
| | | |Reshape{2} [@CC] '' 40
| | | | |_dot22 [@CD] '' 37
| | | | | |Alloc [@CE] '' 34
| | | | | | |TensorConstant{(1, 1) of -1.0} [@CF]
| | | | | | |Shape_i{0} [@BY] '' 32
| | | | | | |Shape_i{1} [@BZ] '' 6
| | | | | |InplaceDimShuffle{1,0} [@CA] 'H.T' 5
| | | | |MakeVector [@BH] '' 18
| | | |Flatten{1} [@Z] '' 2
| | |TensorConstant{0.1} [@CG]
| |CSMProperties{None}.1 [@BP] '' 47
| |CSMProperties{None}.2 [@BP] '' 47
| |Shape [@CH] '' 0
| |W [@G]
|CSMProperties{None}.1 [@BN] '' 49
|CSMProperties{None}.2 [@BN] '' 49
|W [@G]
Stress testing
Forward: 0.205621957779
Forward and Backward: 0.737869024277
![today](https://f.cloud.github.com/assets/335394/91662/ed803650-65a8-11e2-8f53-0dec041a9c8f.png)

(theano-vivek)vvkulkarni@descartes:~/Downloads/graphs_debug_prints_script$