Fix #35704: Simplify on scene with dupli recursion crashes

Couple of issues here:

- User shouldn't be able to run into dupligroup recursion.
  It was checking already when setting a group for dupli.
  Added check to operator which adds object to group.

- It's still possible files with recursion are hanging around,
  so made simplify function robust to such kind of crap.

git-svn-id: https://svn.blender.org/svnroot/bf-blender/trunk/blender@57376 954f8c5b-7b00-dc11-b283-0030488c597c