Close socket on remove
Don't try to read/write data down listener socket