andresy/torch — pull request #117 by akfidjeland (2013-03-13 18:43:24+00:00)
Title: Linear:updateGradInput avoids NaN and inf

The resize in Linear:updateGradInput can introduce NaN and inf into the
gradients. The resize itself leaves garbage in the gradInput tensor.
For normal numbers the subsequent addmm/addmv will clear the garbage.
However, if gradInput contains either nan or inf after the resize, the
multiply will result in nan instead of the desired result.

The following program re-creates the failure

require 'nn'

local function test(x)
local w = nn.Linear(4,4)
w.gradInput:resize(4,4):zero():div(0) -- gradInput is now all NaN
w.gradInput:resize() -- gradInput is now the original size, but will resize back to NaNs
w:updateGradInput(x,x)
assert(torch.ne(w.gradInput, w.gradInput):sum() == 0,
'gradInput contains NaNs. Size' .. tostring(x:size()))
end

test(torch.ones(2,4)) -- minibatch
test(torch.ones(4)) -- stochastic