Fix a bug when connection is closed by both sides.