allow accumulation of gradients, don't use sparse matrix