Yelp/mrjob — Issue #390: max_wait_for_pool option
Author: irskep
Created: 2012-03-08 23:08:23+00:00
State: open
Labels: Feature

Sometimes you occasionally want a job to be cost-effective rather than run exactly when you ask. Pooling can help here: we can add a `--max-wait-for-pool` option to tell a job that it can wait up to X minutes for a free job flow before spinning up a new one. That way there's more flexibility about which pool to take when you have a bunch of jobs running automatically in the middle of the night.