It is a conceptional problem I see. You evict items when someone asks for an item, and instead of returning the existing item, you create a new item, just because the existing item is too old which shouldn't matter with the workers. Instead, you'd need a thread that evicts old workers independent of whether one ask for one. 

Alternatively, you'd need to go to Thread level, use a shared queue and timed take for the request-worker calls, then another queue for the actual tasks.