Fix a problem in a test for developers

* Make it so that other tests won't fail on developer machines after
  ReturnsTrueIfNoNewConfigsAndOneOldConfig runs

Change-Id: I267f1ee363ea447b0e01d45565c28be7fd00a0e0