Only check smart groups that are in use.