Skip work when modifier set is empty.

This both avoids an exception from EnumSet.copyOf when empty and avoids wasting an iterator on looping over an empty set.

Closes #52.