implement sliding next_timeout

Problem: While the worker is processing a job, it polls for image_status
and as part of it, it updates the job.status and job.next_timeout based
on the following params: [job_timeout_update_increment_min,
job_timeout_max_updates]. The timeout will be increased by
job_timeout_update_increment_min up to job_timeout_max_updates number of
times. This can push the timeout out quite a bit (i.e. typically 9
hours) since the job_timeout_update_increment_min is commonly set to 3
hours. Additionally, the timeout isn't udpated until it is expired. This
seems like a risk for allowing another worker to pick up a job that is
still being processed.

It is also worth noting that the timeout can be extended by quite a bit
by the timeout backoff logic that executes in certain error conditions.
This patch does not deal with the backoff logic. It does, however, fix a
broken window and DRY violation nearby.

Fix: This change introduces a new option job_timeout_window_increment
that is used to increase the timeout when the timeout is near
expiration. The worker can increase the timeout up to
job_timeout_max_updates number of times.

Purpose of Sliding Window: An initial timeout is set so that the job can
be picked up by another worker if it is not completed within the
timeout. The sliding window allows the worker to continue working and
keep the job from being picked up by another worker. The reason a single
static timeout is not used is so that if the worker fails in some way
the job will get back into the queue for processing in a more expedient
manner.

RM4877
gimchi lone-rager: the backoff factor is the base
gimchi lone-rager: so if it is 2 then you get powers of 2
gimchi lone-rager: the retries is the exponent
gimchi lone-rager: that is multiplied by increment
lone-rager gimchi: yes, it should go 2,4,8, 16 etc
gimchi lone-rager: correct
gimchi lone-rager: that is multiplied by increment
gimchi lone-rager: it might not be the cleanest, but I don't think it's wrong and we aren't using it are we?
lone-rager gimchi: then why multiply that with increment again?
gimchi lone-rager: no
gimchi lone-rager: let me pull it up real quick not looking at it
gimchi lone-rager: b = backoff_factor ** retry_count
gimchi lone-rager: inc = b * increment
gimchi lone-rager: so that 2, 4, 8, 16 is multiplied by the increment (3 hours originally based on configs from facts-common)
lone-rager gimchi: then timeout = now + inc, right?
gimchi lone-rager: correct
lone-rager gimchi: I say why not timeout = now + b?
gimchi lone-rager: b has no units
gimchi lone-rager: increment has units of time
gimchi lone-rager: the b*increment is applying the backoff to the increment which is your time increment
lone-rager gimchi: yuck, bad naming then
lone-rager gimchi: also, 3 * 4 is half day gone right there
gimchi lone-rager: yep
lone-rager which is only the 2nd try
gimchi lone-rager: yep
gimchi lone-rager: it's not good
lone-rager 3rd try, forget it
gimchi lone-rager: we'll just leave the backoff as 1 and this other patch will address the issue