[quick open] Defer popup destruction to avoid input manager crash

https://bugzilla.gnome.org/show_bug.cgi?id=737711