app: Fix the close button after drag-n-drop

https://bugzilla.gnome.org/show_bug.cgi?id=731998
(cherry picked from commit c4b333658f275b908eb85ce2d6d5a30de221a30c)