Bug 881601 - Don't close the listener if it was never opened, r=wlach