Repository: python-trio/trio

Issue:
SocketStream.__init__'s check for whether the socket is connected is unreliable on MacOS
On IRC, @dstufft told me about a strange crash he sometimes gets in [linehaul](https://github.com/pypa/linehaul) when dumping test data in using netcat, which looked like it might be a bug in trio's `SocketListener` or something.

Unfortunately, I didn't save the traceback, and I'm not really sure how to reproduce it... @dstufft, any idea?


Discussion:
Uhh, I can try to reproduce it again in a bit. I've been focused on Warehouse lately so haven't poked linehaul much. IIRC all I needed to do to repro it was start up linehaul, and use netcat to send 2-3 lines of data through it. As long as I didn't set the flag that kept netcat open I would get the error. Let me see if I can trigger it again.
Ok can reproduce this, it doesn't happen 100% of the time, so I suspect it's some race condition. I just commented out the line that actually sends data to BigQuery, and then ran ``python -m linehaul --log-level debug server --port 8512 --token insecuretoken --credentials-file dev/GCP.json foo.bar.net`` on CPython 3.6.2.

In another termainal, I did ``while true; do netcat localhost 8512 -c < demo; done`` to just continously open up connections, send data 7 lines of data, and then close the connection. Super quickly I got:

```
Traceback (most recent call last):
  File "/Users/dstufft/.pyenv/versions/3.6.2/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/dstufft/.pyenv/versions/3.6.2/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/dstufft/projects/linehaul/linehaul/__main__.py", line 17, in <module>
    cli(prog_name="linehaul")
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/dstufft/projects/linehaul/linehaul/cli.py", line 338, in server
    restrict_keyboard_interrupt_to_checkpoints=True,
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 1267, in run
    return result.unwrap()
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/outcome/_sync.py", line 107, in unwrap
    raise self.error
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 1377, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 962, in init
    self.entry_queue.spawn()
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 274, in asend
    return await self._do_it(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 290, in _do_it
    return await ANextIter(self._it, start_fn, *args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.pyenv/versions/3.6.2/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 426, in _nested_child_finished
    raise MultiError(self._pending_excs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 1377, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/Users/dstufft/projects/linehaul/linehaul/server.py", line 336, in server
    task_status.started()
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 274, in asend
    return await self._do_it(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 290, in _do_it
    return await ANextIter(self._it, start_fn, *args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.pyenv/versions/3.6.2/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 426, in _nested_child_finished
    raise MultiError(self._pending_excs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 1377, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_open_tcp_listeners.py", line 214, in serve_tcp
    task_status=task_status
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_serve_listeners.py", line 129, in serve_listeners
    task_status.started(listeners)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 274, in asend
    return await self._do_it(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 290, in _do_it
    return await ANextIter(self._it, start_fn, *args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.pyenv/versions/3.6.2/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 200, in open_cancel_scope
    yield scope
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 315, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 426, in _nested_child_finished
    raise MultiError(self._pending_excs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_core/_run.py", line 1377, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_serve_listeners.py", line 36, in _serve_one_listener
    stream = await listener.accept()
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_socket.py", line 384, in accept
    return SocketStream(sock)
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_socket.py", line 70, in __init__
    raise err from None
ValueError: SocketStream requires a connected socket
```


My uneducated guess is that because netcat is only sending 7 lines, and because it's closing the connection immediately after is that it's closing the connection before trio has a chance to actually accept the connection?
What happens when you remove the `getpeername` call (delete `trio/trio/_highlevel_socket.py`:66-70 ?
… no, that's not it, testing reveals that `getpeername()` works quite well on a half-closed socket.
I can't reproduce this.
Can you run the linehaul command under `strace` and put that into some pastebin or other?
```
  File "/Users/dstufft/.virtualenvs/linehaul-cpython/lib/python3.6/site-packages/trio/_highlevel_socket.py", line 70, in __init__
    raise err from None
```

Temporarily getting rid of the `from None` there might also help shed some light on the matter.
OK, so the issue is that MacOS is weird:

```
>>> import sys
>>> sys.platform
'darwin'
>>> import socket
>>> a, b = socket.socketpair()
>>> a.getpeername()
''
>>> b.close()
>>> a.getpeername()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
```

This is sorta-documented in the getpeername(2) man page:

>      [EINVAL]           socket has been shut down.

This is pretty annoying... on MacOS apparently you can have a situation where you accept a connection, read data from it, everything works fine... except that it is literally impossible to find out who you are talking to, because the kernel knows that but threw away the information before it let you start seeing the data.

Oh well, it is what it is.

So I guess we... ignore EINVAL when checking the socket in `SocketStream.__init__`? On Linux the man page says EINVAL only happens if the "addrlen is invalid", which should never happen if Python's `getpeername` is correctly written, and indeed I didn't manage to get an EINVAL when playing around a bit – e.g. calling it on a non-existent fd gives `EBADF`, and on a non-socket fd gives `ENOTSOCK`.

Alternatively, I guess we could remove the connected check altogether? If you try to wrap a not-connected socket in `SocketStream` then you're going to get some explosions soon enough...
Marking this a good first issue. I think the items to do are:

1. Add a test that demonstrates the bug. Something like:

```python
async def test_socket_stream_works_when_peer_has_already_closed():
    a, b = trio.socket.socketpair()
    await b.send(b"x")
    b.close()
    stream = trio.SocketStream(a)
    assert stream.receive_some(1) == b"x"
    assert stream.receive_some(1) == b""
```

I believe the above test will currently fail on MacOS.

2. Make the test pass by deleting the `getpeername` check from `SocketStream.__init__`.

3. The normal [contributing a PR stuff](https://trio.readthedocs.io/en/latest/contributing.html#preparing-pull-requests), like adding a news entry to the `newsfragments/` directory.
> Alternatively, I guess we could remove the connected check altogether?

Probably the best way of (not-)handling this issue.
> I believe the above test will currently fail on MacOS.

Confirmed. Fails with `ValueError: SocketStream requires a connected socket`.

Resolving patch:
diff --git a/trio/_highlevel_socket.py b/trio/_highlevel_socket.py
--- a/trio/_highlevel_socket.py
+++ b/trio/_highlevel_socket.py
@@ -62,11 +62,6 @@ def __init__(self, socket):
             raise TypeError("SocketStream requires trio socket object")
         if socket.type != tsocket.SOCK_STREAM:
             raise ValueError("SocketStream requires a SOCK_STREAM socket")
-        try:
-            socket.getpeername()
-        except OSError:
-            err = ValueError("SocketStream requires a connected socket")
-            raise err from None
 
         self.socket = socket
         self._send_conflict_detector = ConflictDetector(


Test patch:
diff --git a/trio/tests/test_highlevel_open_tcp_stream.py b/trio/tests/test_highlevel_open_tcp_stream.py
--- a/trio/tests/test_highlevel_open_tcp_stream.py
+++ b/trio/tests/test_highlevel_open_tcp_stream.py
@@ -128,9 +128,6 @@ def close(self):
     def setsockopt(self, *args, **kwargs):
         pass
 
-    def getpeername(self):
-        return (self.ip, self.port)
-
 
 class Scenario(trio.abc.SocketFactory, trio.abc.HostnameResolver):
     def __init__(self, port, ip_list, ipv6_supported):
diff --git a/trio/tests/test_highlevel_socket.py b/trio/tests/test_highlevel_socket.py
--- a/trio/tests/test_highlevel_socket.py
+++ b/trio/tests/test_highlevel_socket.py
@@ -24,11 +24,6 @@ async def test_SocketStream_basics():
         with pytest.raises(ValueError):
             SocketStream(sock)
 
-    # disconnected socket bad
-    with tsocket.socket() as sock:
-        with pytest.raises(ValueError):
-            SocketStream(sock)
-
     a, b = tsocket.socketpair()
     with a, b:
         s = SocketStream(a)
@@ -220,10 +215,6 @@ def getsockopt(self, level, opt):
         def setsockopt(self, level, opt, value):
             pass
 
-        # Fool the check for connection in SocketStream.__init__
-        def getpeername(self):
-            pass
-
         async def accept(self):
             await _core.checkpoint()
             event = next(self._events)
@@ -262,3 +253,12 @@ async def accept(self):
     with assert_checkpoints():
         s = await l.accept()
         assert s.socket is fake_server_sock
+
+
+async def test_socket_stream_works_when_peer_has_already_closed():
+    sock_a, sock_b = tsocket.socketpair()
+    await sock_b.send(b"x")
+    sock_b.close()
+    stream = SocketStream(sock_a)
+    assert await stream.receive_some(1) == b"x"
+    assert await stream.receive_some(1) == b""