agent: Don't hold lock while invoking handlers