Compute the gradient for CoFi