By default, Salt will remove any groups not listed, so a users groups matches exactly the list you pass.  This was added here:
https://github.com/saltstack/salt/issues/2142

This has been causing issues for many people, as the remove_groups options is undocumented.  In the 2014.7 release this is changing, and remove_groups will default to false:
https://github.com/saltstack/salt/issues/13276

I'm going with false by default, as it's our use case and it will soon be the default.  If people believe this module should default to true and remove groups not listed, I think that's open for discussion, but we should at least add the option.
Make sure the logic stands after the default in salt is changed
Merge pull request #54 from JaseFace/master

Add options for remove_groups and default to false
Sorry for the spam, simplify this remove_groups rule a bit
Merge pull request #55 from JaseFace/master

Sorry for the spam, simplify this remove_groups rule a bit
Remove trailing slash from sudoers_dir
Merge pull request #56 from Cottser/sudoers-dir-trailing-slash

Remove trailing slash from sudoers_dir