Fixes flaky no-such-element exception from bad generation of random tests.
Ok, this is actually the flaky issue with the test.

We use the ch key for testing, so it can't be part of the autogenerated set.
Merge pull request #1521 from sbt/wip/fix-flaky-settings-test

Fixes flaky no-such-element exception from bad generation of random test...