Repository: cherrypy/cheroot

Issue:
Add an option for port re-using

❓ **I'm submitting a ...**
- [ ] 🐞 bug report
- [X] 🐣 feature request
- [ ] ❓ question about the decisions made in the repository

🐞 **Is your feature request related to a problem? Please describe.**
We have a process that automatically distributes available ports across multiple cheroot processes and keeps sockets on those ports opened until cheroot processes take control on them. Cheroot right now cannot reuse those opened ports.


🐣 **Describe the solution you'd like**
Add an option to say cheroot to set SO_REUSEPORT socket option.


📋 **Describe alternatives you've considered**
Passing socket object to cheroot instead of bind address, but that would require some refactoring and I don't think it would follow Zen of Cherrypy if this solution would be ported to cherrypy.



Discussion:
VladimirKuzmin: In my opinion here is the key logical difference between `SO_REUSEADDR` and `SO_REUSEPORT` (with exception to Windows):
* `SO_REUSEADDR` is used to resolve conflict with some unrelated processes, and can be set by a process listening to non-wildcard addresses;
* `SO_REUSEPORT` is used as an tool for system design (load balancing, assigning ports by some coordinating process, picking up port from another stuck process) and it's set intentionally by every process using the address.

Turning it on by default can lead to a situation when two independent processes (e.g two different cherrypy/cheroot servers) accidentally open connections to the same port and address, set `SO_REUSEPORT` just in case and get only part of their requests. It might be a rare case, but it would be hard to debug. So I am still inclined to use the extra parameter

I will add better handling of some special cases:
* Windows can use `SO_REUSEADDR` instead of `SO_REUSEPORT`;
* Ephemeral socket should not be used together with reuse_port, will add a check or switch off `SO_REUSEPORT` if we go with the solution without the extra parameter;
* Will do research on abstract sockets.
VladimirKuzmin: Sorry for the long delay.  I updated my PR and resolved previously discussed issues.

Also, here are some examples of port reuse implementations in Python web servers:
https://github.com/benoitc/gunicorn/blob/20.1.0/gunicorn/sock.py#L41
https://github.com/unbit/uwsgi/blob/2.0.21/core/socket.c#L699
https://github.com/jonashaag/bjoern/blob/3.2.2/bjoern.py#L24

Alternatively, if this feature is too expensive to maintain, I can send PR making `prepare_socket` a class method instead of a static method, so that method can be overridden.

Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index ee7becc6..cf19e8ef 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,3 +1,10 @@
+.. scm-version-title:: v10.0.0
+
+- :issue:`504` via :pr:`505`: Cheroot now accepts a
+  ``reuse_port`` parameter on the ``HTTPServer`` object.
+  Subclasses overriding ``prepare_socket`` will no longer
+  work and will need to adapt to the new interface.
+
 .. scm-version-title:: v9.0.0
 
 - :issue:`252` via :pr:`339`: Cheroot now requires Python
diff --git a/cheroot/server.py b/cheroot/server.py
index 13977222..bceeb2c9 100644
--- a/cheroot/server.py
+++ b/cheroot/server.py
@@ -1572,6 +1572,9 @@ class HTTPServer:
     ``PEERCREDS``-provided IDs.
     """
 
+    reuse_port = False
+    """If True, set SO_REUSEPORT on the socket."""
+
     keep_alive_conn_limit = 10
     """Maximum number of waiting keep-alive connections that will be kept open.
 
@@ -1581,6 +1584,7 @@ class HTTPServer:
         self, bind_addr, gateway,
         minthreads=10, maxthreads=-1, server_name=None,
         peercreds_enabled=False, peercreds_resolve_enabled=False,
+        reuse_port=False,
     ):
         """Initialize HTTPServer instance.
 
@@ -1591,6 +1595,8 @@ class HTTPServer:
             maxthreads (int): maximum number of threads for HTTP thread pool
             server_name (str): web server name to be advertised via Server
                 HTTP header
+            reuse_port (bool): if True SO_REUSEPORT option would be set to
+                socket
         """
         self.bind_addr = bind_addr
         self.gateway = gateway
@@ -1606,6 +1612,7 @@ class HTTPServer:
         self.peercreds_resolve_enabled = (
             peercreds_resolve_enabled and peercreds_enabled
         )
+        self.reuse_port = reuse_port
         self.clear_stats()
 
     def clear_stats(self):
@@ -1880,6 +1887,7 @@ class HTTPServer:
             self.bind_addr,
             family, type, proto,
             self.nodelay, self.ssl_adapter,
+            self.reuse_port,
         )
         sock = self.socket = self.bind_socket(sock, self.bind_addr)
         self.bind_addr = self.resolve_real_bind_addr(sock)
@@ -1928,6 +1936,7 @@ class HTTPServer:
             bind_addr=bind_addr,
             family=socket.AF_UNIX, type=socket.SOCK_STREAM, proto=0,
             nodelay=self.nodelay, ssl_adapter=self.ssl_adapter,
+            reuse_port=self.reuse_port,
         )
 
         try:
@@ -1968,7 +1977,36 @@ class HTTPServer:
         return sock
 
     @staticmethod
-    def prepare_socket(bind_addr, family, type, proto, nodelay, ssl_adapter):
+    def _make_socket_reusable(socket_, bind_addr):
+        host, port = bind_addr[:2]
+        IS_EPHEMERAL_PORT = port == 0
+
+        if socket_.family not in (socket.AF_INET, socket.AF_INET6):
+            raise ValueError('Cannot reuse a non-IP socket')
+
+        if IS_EPHEMERAL_PORT:
+            raise ValueError('Cannot reuse an ephemeral port (0)')
+
+        # Most BSD kernels implement SO_REUSEPORT the way that only the
+        # latest listener can read from socket. Some of BSD kernels also
+        # have SO_REUSEPORT_LB that works similarly to SO_REUSEPORT
+        # in Linux.
+        if hasattr(socket, 'SO_REUSEPORT_LB'):
+            socket_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT_LB, 1)
+        elif hasattr(socket, 'SO_REUSEPORT'):
+            socket_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+        elif IS_WINDOWS:
+            socket_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        else:
+            raise NotImplementedError(
+                'Current platform does not support port reuse',
+            )
+
+    @classmethod
+    def prepare_socket(
+            cls, bind_addr, family, type, proto, nodelay, ssl_adapter,
+            reuse_port=False,
+    ):
         """Create and prepare the socket object."""
         sock = socket.socket(family, type, proto)
         connections.prevent_socket_inheritance(sock)
@@ -1976,6 +2014,9 @@ class HTTPServer:
         host, port = bind_addr[:2]
         IS_EPHEMERAL_PORT = port == 0
 
+        if reuse_port:
+            cls._make_socket_reusable(socket_=sock, bind_addr=bind_addr)
+
         if not (IS_WINDOWS or IS_EPHEMERAL_PORT):
             """Enable SO_REUSEADDR for the current socket.
 
diff --git a/cheroot/server.pyi b/cheroot/server.pyi
index 864adff4..ecbe2f27 100644
--- a/cheroot/server.pyi
+++ b/cheroot/server.pyi
@@ -130,9 +130,10 @@ class HTTPServer:
     ssl_adapter: Any
     peercreds_enabled: bool
     peercreds_resolve_enabled: bool
+    reuse_port: bool
     keep_alive_conn_limit: int
     requests: Any
-    def __init__(self, bind_addr, gateway, minthreads: int = ..., maxthreads: int = ..., server_name: Any | None = ..., peercreds_enabled: bool = ..., peercreds_resolve_enabled: bool = ...) -> None: ...
+    def __init__(self, bind_addr, gateway, minthreads: int = ..., maxthreads: int = ..., server_name: Any | None = ..., peercreds_enabled: bool = ..., peercreds_resolve_enabled: bool = ..., reuse_port: bool = ...) -> None: ...
     stats: Any
     def clear_stats(self): ...
     def runtime(self): ...
@@ -152,7 +153,9 @@ class HTTPServer:
     def bind(self, family, type, proto: int = ...): ...
     def bind_unix_socket(self, bind_addr): ...
     @staticmethod
-    def prepare_socket(bind_addr, family, type, proto, nodelay, ssl_adapter): ...
+    def _make_socket_reusable(socket_, bind_addr) -> None: ...
+    @classmethod
+    def prepare_socket(cls, bind_addr, family, type, proto, nodelay, ssl_adapter, reuse_port: bool = ...): ...
     @staticmethod
     def bind_socket(socket_, bind_addr): ...
     @staticmethod
diff --git a/cheroot/wsgi.py b/cheroot/wsgi.py
index 82faca3e..1dbe10ee 100644
--- a/cheroot/wsgi.py
+++ b/cheroot/wsgi.py
@@ -43,6 +43,7 @@ class Server(server.HTTPServer):
         max=-1, request_queue_size=5, timeout=10, shutdown_timeout=5,
         accepted_queue_size=-1, accepted_queue_timeout=10,
         peercreds_enabled=False, peercreds_resolve_enabled=False,
+        reuse_port=False,
     ):
         """Initialize WSGI Server instance.
 
@@ -69,6 +70,7 @@ class Server(server.HTTPServer):
             server_name=server_name,
             peercreds_enabled=peercreds_enabled,
             peercreds_resolve_enabled=peercreds_resolve_enabled,
+            reuse_port=reuse_port,
         )
         self.wsgi_app = wsgi_app
         self.request_queue_size = request_queue_size
diff --git a/cheroot/wsgi.pyi b/cheroot/wsgi.pyi
index 96075633..f96a18f9 100644
--- a/cheroot/wsgi.pyi
+++ b/cheroot/wsgi.pyi
@@ -8,7 +8,7 @@ class Server(server.HTTPServer):
     timeout: Any
     shutdown_timeout: Any
     requests: Any
-    def __init__(self, bind_addr, wsgi_app, numthreads: int = ..., server_name: Any | None = ..., max: int = ..., request_queue_size: int = ..., timeout: int = ..., shutdown_timeout: int = ..., accepted_queue_size: int = ..., accepted_queue_timeout: int = ..., peercreds_enabled: bool = ..., peercreds_resolve_enabled: bool = ...) -> None: ...
+    def __init__(self, bind_addr, wsgi_app, numthreads: int = ..., server_name: Any | None = ..., max: int = ..., request_queue_size: int = ..., timeout: int = ..., shutdown_timeout: int = ..., accepted_queue_size: int = ..., accepted_queue_timeout: int = ..., peercreds_enabled: bool = ..., peercreds_resolve_enabled: bool = ..., reuse_port: bool = ...) -> None: ...
     @property
     def numthreads(self): ...
     @numthreads.setter


Test patch:
diff --git a/cheroot/test/test_server.py b/cheroot/test/test_server.py
index 92fc31f8..e2e5f788 100644
--- a/cheroot/test/test_server.py
+++ b/cheroot/test/test_server.py
@@ -373,6 +373,33 @@ def test_high_number_of_file_descriptors(native_server_client, resource_limit):
     assert any(fn >= resource_limit for fn in native_process_conn.filenos)
 
 
+@pytest.mark.skipif(
+    not hasattr(socket, 'SO_REUSEPORT'),
+    reason='socket.SO_REUSEPORT is not supported on this platform',
+)
+@pytest.mark.parametrize(
+    'ip_addr',
+    (
+        ANY_INTERFACE_IPV4,
+        ANY_INTERFACE_IPV6,
+    ),
+)
+def test_reuse_port(http_server, ip_addr, mocker):
+    """Check that port initialized externally can be reused."""
+    family = socket.getaddrinfo(ip_addr, EPHEMERAL_PORT)[0][0]
+    s = socket.socket(family)
+    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+    s.bind((ip_addr, EPHEMERAL_PORT))
+    server = HTTPServer(
+        bind_addr=s.getsockname()[:2], gateway=Gateway, reuse_port=True,
+    )
+    spy = mocker.spy(server, 'prepare')
+    server.prepare()
+    server.stop()
+    s.close()
+    assert spy.spy_exception is None
+
+
 ISSUE511 = IS_MACOS