Remove the group on this directory.