Make sure empty group or user are not created when editing