Factor gradients out of Query