Do not apply implicit group assignment for other instance badges
Fix user badge assignments

Clean up code and make sure that only shown badges are removed.