Configurable per-pipeline delayed start of workers