Only cache points if there are no bounds.