Refactor runinboud/runnightly modules

Unrelated to this change, I feel like we should actually throw an assertion/exception in this case since we can't honor the user's choice.