Add settings to throttle scheduled runs

Change https://gerrit.wikimedia.org/r/#/c/142007/ is implementing this
in the wikimetrics codebase.

Change-Id: Ia3277788a074e8476927583af4db5004a7a70a5e