[KBSwitch] Don't set groups/variants if it's empty.

Fixes #1590.