More logical locking with workers (reading often)

Signed-off-by: Robert Xu <robxu9@gmail.com>