Repository: HangfireIO/Hangfire

Issue #595: Recurring job '?' can not be scheduled due to job load exception.
Created: 2016-06-15T20:10:25Z
Talking about Scalability/Distrubted Systems I think each server might see only its queues.
In my production envrionment I'm running two instances in two servers. Each server actually runs over different queues as below

Server "A" has only libraries needed there and in the other server ("B") only the libraries needed also. When server "B" tries to Schedule a job it throws JobLoadException because there's no library into its binary folder. What I suggest is to change few classes to restrict RecurringJobScheduler class to watch only the right queues.
The queue is already in the context and could be used to filter recurring-jobs as well.

Duplicate issue #910: Hangfire cannot work when servers running on different code bases use the same database
Created: 2017-06-02T20:12:20Z
This issue is somewhat related to issue #908, but provides more details and possible fix.
Short description
In a case when multiple instances of Hangfire use the same database, but code is different, Hangfire servers may fail to instantiate jobs. In this case job is rescheduled.
For instance if I have servers A.v1 and A.v2 and I have JobV1 on A.v1 and JobV2 on A.v2, server A.v1 eventually may try to en-queue JobV2 and fail on de-serialization as codebase is different.
Queues cannot help to solve this problem as queue information is attached to the job and it is the job that cannot be instantiated. If I have ten different servers the job have 10% chance to be re-queued sucesfully.
Why it matters
We have quite a few distinct micro-servers which use Hangfire. Let's say we have ten types of servers and they deployed into eight geographic zones in two environments (test and prod). Today we have 16 separate Hangfire databases - one per zone per environment (32 during roll-out). Today we forced to do synchronous deployment to all servers in one zone and environment whenever code for any of the job changes.  We also forced to keep all the jobs in a separate common project.  This is a real headache.
If we want to separate codebase per server type and stop maintaining common job project we looking into maintaining 160 instances of Hangfire databases (actually 320 during the roll-out unless it is synchronous)
Possible solution
When Hangfire selects job for execution in DelayedJobScheduler it specifies key parameter to GetFirstByLowestScoreFromSet, which is always "schedule".
In fact Hangfire uses that "schedule" key exactly in four spots: when background job is selected, removed or added to the set.
We propose to change the notion of the key, so it defines not only the job status ("schedule","recurring-jobs",etc) but also a codebase, for instance  "schedule_orderManagement_v1.10"
When application starts it specifies (optional) codebase, like:
_GlobalConfiguration.Configuration.UseCodeBase("orderManagement_v1.10")
All places in the code which access the set should use codebase suffix, which is empty by default and existing functionality is completely unchanged (including queues etc).
Now I can deploy new version of the code to any instance of micro-server in my zone and be sure that other Hangfire instances will not to try to work with the job they cannot instantiate
We have the fix for  "schedule" case right now and actively testing it.

Closing comment (2020-05-18T07:49:09Z): Duplicate of #595