x11: close both of the display's we use

(the comment doesn't seem to apply anymore)

https://bugzilla.gnome.org/show_bug.cgi?id=725048
Merge branch 'master' into ci