Repository: python-trio/trio

Issue:
Add a method to trio socket objects that checks whether the OS thinks the socket is readable
Say you're writing an HTTP/1.1 client library, that supports connection pooling. When you take an idle connection out of your pool, it might be fine to use... OR, the server might have gotten tired of waiting for that connection to do something while it was sitting idle in the pool, and have already closed it. Because of infelicities in HTTP/1.1's design, there is no 100% reliable way to detect this, because there's a race condition: you might send a new request down the connection just as the server is closing it, and then you have no idea what happened and whether the connection dropped before or after the server started processing your request.

So, we would like to minimize how often this happens. But, because there's actually no way to detect it at the protocol level, there really is no good way to do this using regular high-level APIs. The best approach is: when you take an idle connection out of the pool, before you start using it again, do a quick check whether the underlying socket is readable. You don't want to actually read from it; there's nothing to learn by doing so. If an HTTP/1.1 server sent anything on an idle connection, it's either a 408 Request Timeout or just a preemptory close. You especially don't want to block waiting to receive data, since that would defeat the whole purpose – we just want to check whether you *could* receive data. And you want to do this directly on the raw socket, ignoring things like any TLS encryption layer, because TLS does not really provide any way to do a pure check for whether *unencrypted* bytes are available – to figure that out you first have to read the encrypted bytes and then check if they make a complete frame, and we don't want to read any bytes.

So... for this specific, weird, and incredibly important use case, you don't want to use Trio's high-level `Stream` layer, you want to peek through all the abstraction layers, ignore Trio's socket I/O API entirely, and call `select` or `poll` directly on the underlying socket just to ask the OS whether it is readable.

So far, so good; Trio lets you do all that using supported, public APIs.

But! Suppose that for testing, you want to use a fake virtual networking layer. That's great: we have first-class support for that in Trio's socket I/O API (#170). Except... asks or urllib3 or whatever, for this one operation, currently has to do an end-run around Trio's socket I/O API – it has to directly call the OS's `select` or `poll` operation, which means that it needs a real OS socket, not a fake virtual socket. And that means that currently, our network faking API cannot handle HTTP clients, which is kind of an important use case.

Solution: we should add a method on Trio socket objects that does this readability check. It's pretty straightforward, [though there are a few complications](https://github.com/urllib3/urllib3/blob/master/src/urllib3/util/wait.py). (Mostly, you need to use `select` on windows and `poll` everywhere else.) And the whole point of this is that libraries like asks or urllib3 will call this method instead of doing their own call to `select`/`poll`. And it will do exactly the same thing they would have done by hand. Except..... since it's a method on the socket object, our fake virtual sockets will be able to override it to do a fake virtual `select`/`poll`. And that will solve the problem.


Discussion:
@wgwz You mentioned in gitter that you had some questions about this?
This is slightly larger than we usually use for "good first issues", but if you're ambitious then it could work.

Basically what's needed is:

- a new method `is_readable` on `trio._socket._SocketType`
  - on Windows, it does `rready, _, _ = select.select([self._sock], [], [], 0); return bool(rready)`
  - everywhere else, it does `p = select.poll(); p.register(self._sock, select.POLLIN); return bool(p.poll(0))`
  - don't actually write the code as a one-liner, use normal formatting :-)
- a new test in `trio/tests/test_socket.py`
- add it to the docs
- add a [newfragment](https://github.com/python-trio/trio/blob/master/newsfragments/README.rst)
I'll have a look at this

Resolving patch:
diff --git a/trio/_socket.py b/trio/_socket.py
--- a/trio/_socket.py
+++ b/trio/_socket.py
@@ -1,5 +1,6 @@
 import os as _os
 import sys as _sys
+import select
 import socket as _stdlib_socket
 from functools import wraps as _wraps
 
@@ -289,7 +290,7 @@ def socket(
 
 def _sniff_sockopts_for_fileno(family, type, proto, fileno):
     """Correct SOCKOPTS for given fileno, falling back to provided values.
-    
+
     """
     # Wrap the raw fileno into a Python socket object
     # This object might have the wrong metadata, but it lets us easily call getsockopt
@@ -478,6 +479,15 @@ def shutdown(self, flag):
         if flag in [_stdlib_socket.SHUT_WR, _stdlib_socket.SHUT_RDWR]:
             self._did_shutdown_SHUT_WR = True
 
+    def is_readable(self):
+        # use select.select on Windows, and select.poll everywhere else
+        if _sys.platform == "win32":
+            rready, _, _ = select.select([self._sock], [], [], 0)
+            return bool(rready)
+        p = select.poll()
+        p.register(self._sock, select.POLLIN)
+        return bool(p.poll(0))
+
     async def wait_writable(self):
         await _core.wait_writable(self._sock)
 


Test patch:
diff --git a/trio/tests/test_socket.py b/trio/tests/test_socket.py
--- a/trio/tests/test_socket.py
+++ b/trio/tests/test_socket.py
@@ -404,6 +404,17 @@ async def test_SocketType_simple_server(address, socket_type):
             assert await client.recv(1) == b"x"
 
 
+async def test_SocketType_is_readable():
+    a, b = tsocket.socketpair()
+    with a, b:
+        assert not a.is_readable()
+        await b.send(b"x")
+        await _core.wait_readable(a)
+        assert a.is_readable()
+        assert await a.recv(1) == b"x"
+        assert not a.is_readable()
+
+
 # On some macOS systems, getaddrinfo likes to return V4-mapped addresses even
 # when we *don't* pass AI_V4MAPPED.
 # https://github.com/python-trio/trio/issues/580