Allow that inner_product accumulates gradients