Hold active tasks in a map, but cache finished tasks briefly