Avoid holding the lock while scanning for existing jobs

Under python we can safely check if a string key is in
a dictionary without having to hold a lock on that dictionary
to avoid concurrent deletions from affecting this check,
so we can use that to our benefit to avoid holding the lock
when searching for which potential jobs to request that do
not already exist in the currently known jobs.

Change-Id: I26c3bed2800789720cfc907c8e7dcbb3bad36447