Fix scope issue where higher index users are replaced by lower ones when they are removed