[6.6] Fixes a bug with group member administration

Group owners were not able to remove any users from their group if they were the only owner.

Pretty easy to reproduce.  Create a group then add someone else to it with any permissions except Owner.  Notice you will not be able to remove them.

This PR is into master, will open another for 6-6-stable if desired.