Do not fetch twice for LDAP/AD groups as well