Hold active tasks in a map, but cache finished tasks briefly

I think this might need to also look in the finished tasks. A client will keep asking for a key, and after it's finished it should return the finished handle.