Set use_for_related_fields=True on QueryManager.