Remove group when the elements don't match the filter condition anymore

And also better support for custom GroupModel