Keep hovered gradient direction the same with respect to what we're
hovering over.