Move canonical run configuration objects to a sub-package

This depends on #4083, please review that first.