Fix incorrect SetParent calls
fix error when dealing with diagonal matrix groups

also added a test case for that
Fix error leading to an endless loop

While I have no easily reproducible test case, I kept running
into this in my work. The change here fixes it for me.