xcb: Remove display from attached list on free

Signed-off-by: Quentin Glidic <sardemff7+git@sardemff7.net>