remove the group in case the last user has removed from that group