Repository: python-trio/trio

Issue:
Add CI for FreeBSD (and other BSDs?)
See https://cirrus-ci.org/guide/FreeBSD/ for details.


Discussion:
Sounds fine to me.
For the record: I "installed" the Cirrus-CI app on the python-trio org, at @thedrow's request. So I think this just needs someone to put together a PR with a `.cirrus.yml` in it.

Our general strategy for these CI systems is to try to keep the logic in the `.yml` file to the bare minimum, and do everything else through `ci.sh` (which is shared across CI platforms).
There's some more discussion of Cirrus here: https://github.com/python-trio/trio/pull/1128

Summary:

- The trio tests have some problems on freebsd, so that needs to be sorted out before we could enable CI testing
- Cirrus works, but is slow and we ran into some bugs even on just this first test PR, which isn't very confidence inspiring.

It's possible that Cirrus has gotten better since then, or will by the time anyone gets around to fixing the tests.

There are also some other options for BSD testing though:

- We can use nested virtualization on Travis to run tests on arbitrary OSes. Currently we're using this to test our support for cutting edge Linux kernel APIs, but it would work for BSDs too.

  This is also kind of slow. It might or might not be faster than Cirrus; we'd have to check. It's definitely a bit complicated to set up.

- We could use SourceHut's "builds" feature, which has full support for BSDs and GitHub: https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/ (thanks to @wgwz for bringing this to my attention)

  Unlike the other options here, this is technically a paid service. It's a tiny self-funded operation, so they ask that somebody pay something like $50-100/year to use it: https://sourcehut.org/pricing/ They're really not making any money off it, and we could test it out for free before committing.

Anyway, if someone wants to push this forward, the first step is just to get the tests passing.
NB: github actions currently also supports nested virtualization *but* only on their macOS runners.

Resolving patch:
diff --git a/notes-to-self/fbsd-pipe-close-notify.py b/notes-to-self/fbsd-pipe-close-notify.py
new file mode 100644
--- /dev/null
+++ b/notes-to-self/fbsd-pipe-close-notify.py
@@ -0,0 +1,38 @@
+# This script completes correctly on macOS and FreeBSD 13.0-CURRENT, but hangs
+# on FreeBSD 12.1. I'm told the fix will be backported to 12.2 (which is due
+# out in October 2020).
+#
+# Upstream bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246350
+
+import select
+import os
+import threading
+
+r, w = os.pipe()
+
+os.set_blocking(w, False)
+
+print("filling pipe buffer")
+while True:
+    try:
+        os.write(w, b"x")
+    except BlockingIOError:
+        break
+
+_, wfds, _ = select.select([], [w], [], 0)
+print("select() says the write pipe is", "writable" if w in wfds else "NOT writable")
+
+kq = select.kqueue()
+event = select.kevent(w, select.KQ_FILTER_WRITE, select.KQ_EV_ADD)
+kq.control([event], 0)
+
+print("closing read end of pipe")
+os.close(r)
+
+_, wfds, _ = select.select([], [w], [], 0)
+print("select() says the write pipe is", "writable" if w in wfds else "NOT writable")
+
+print("waiting for kqueue to report the write end is writable")
+got = kq.control([], 1)
+print("done!")
+print(got)
diff --git a/trio/_core/_io_kqueue.py b/trio/_core/_io_kqueue.py
--- a/trio/_core/_io_kqueue.py
+++ b/trio/_core/_io_kqueue.py
@@ -3,6 +3,7 @@
 import outcome
 from contextlib import contextmanager
 import attr
+import errno
 
 from .. import _core
 from ._run import _public
@@ -122,16 +123,24 @@ def abort(_):
             event = select.kevent(fd, filter, select.KQ_EV_DELETE)
             try:
                 self._kqueue.control([event], 0)
-            except FileNotFoundError:
+            except OSError as exc:
                 # kqueue tracks individual fds (*not* the underlying file
                 # object, see _io_epoll.py for a long discussion of why this
                 # distinction matters), and automatically deregisters an event
                 # if the fd is closed. So if kqueue.control says that it
                 # doesn't know about this event, then probably it's because
-                # the fd was closed behind our backs. (Too bad it doesn't tell
-                # us that this happened... oh well, you can't have
-                # everything.)
-                pass
+                # the fd was closed behind our backs. (Too bad we can't ask it
+                # to wake us up when this happens, versus discovering it after
+                # the fact... oh well, you can't have everything.)
+                #
+                # FreeBSD reports this using EBADF. macOS uses ENOENT.
+                if exc.errno in (
+                    errno.EBADF, errno.ENOENT
+                ):  # pragma: no branch
+                    pass
+                else:  # pragma: no cover
+                    # As far as we know, this branch can't happen.
+                    raise
             return _core.Abort.SUCCEEDED
 
         await self.wait_kevent(fd, filter, abort)
diff --git a/trio/socket.py b/trio/socket.py
--- a/trio/socket.py
+++ b/trio/socket.py
@@ -106,7 +106,8 @@
         CAN_BCM_TX_ANNOUNCE, CAN_BCM_TX_COUNTEVT, CAN_BCM_TX_CP_CAN_ID,
         CAN_BCM_TX_RESET_MULTI_IDX, IPPROTO_CBT, IPPROTO_ICLFXBM, IPPROTO_IGP,
         IPPROTO_L2TP, IPPROTO_PGM, IPPROTO_RDP, IPPROTO_ST, AF_QIPCRTR,
-        CAN_BCM_CAN_FD_FRAME
+        CAN_BCM_CAN_FD_FRAME, IPPROTO_MOBILE, IPV6_USE_MIN_MTU,
+        MSG_NOTIFICATION, SO_SETFIB
     )
 except ImportError:
     pass


Test patch:
diff --git a/trio/_core/tests/tutil.py b/trio/_core/tests/tutil.py
--- a/trio/_core/tests/tutil.py
+++ b/trio/_core/tests/tutil.py
@@ -1,5 +1,6 @@
 # Utilities for testing
 import socket as stdlib_socket
+import os
 
 import pytest
 
@@ -61,3 +62,11 @@ def check_sequence_matches(seq, template):
         got = set(seq[i:i + len(pattern)])
         assert got == pattern
         i += len(got)
+
+
+# https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246350
+skip_if_fbsd_pipes_broken = pytest.mark.skipif(
+    hasattr(os, "uname") and os.uname().sysname == "FreeBSD"
+    and os.uname().release[:4] < "12.2",
+    reason="hangs on FreeBSD 12.1 and earlier, due to FreeBSD bug #246350"
+)
diff --git a/trio/tests/test_highlevel_open_tcp_listeners.py b/trio/tests/test_highlevel_open_tcp_listeners.py
--- a/trio/tests/test_highlevel_open_tcp_listeners.py
+++ b/trio/tests/test_highlevel_open_tcp_listeners.py
@@ -74,8 +74,16 @@ async def measure_backlog(listener, limit):
             # the SYN gets dropped, and the client retries after 1 second. So
             # we assume that any connect() call to localhost that takes >0.5
             # seconds indicates a dropped SYN.
+            #
+            # Exception: on FreeBSD when the backlog is exhausted, connect
+            # has been observed to sometimes raise ConnectionResetError.
             with trio.move_on_after(0.5) as cancel_scope:
-                client_stream = await open_stream_to_socket_listener(listener)
+                try:
+                    client_stream = await open_stream_to_socket_listener(
+                        listener
+                    )
+                except ConnectionResetError:  # pragma: no cover
+                    break
                 client_streams.append(client_stream)
             if cancel_scope.cancelled_caught:
                 break
diff --git a/trio/tests/test_subprocess.py b/trio/tests/test_subprocess.py
--- a/trio/tests/test_subprocess.py
+++ b/trio/tests/test_subprocess.py
@@ -9,7 +9,7 @@
     _core, move_on_after, fail_after, sleep, sleep_forever, Process,
     open_process, run_process, TrioDeprecationWarning
 )
-from .._core.tests.tutil import slow
+from .._core.tests.tutil import slow, skip_if_fbsd_pipes_broken
 from ..testing import wait_all_tasks_blocked
 
 posix = os.name == "posix"
@@ -254,6 +254,7 @@ async def test_run_check():
     assert result.returncode == 1
 
 
+@skip_if_fbsd_pipes_broken
 async def test_run_with_broken_pipe():
     result = await run_process(
         [sys.executable, "-c", "import sys; sys.stdin.close()"],
diff --git a/trio/tests/test_unix_pipes.py b/trio/tests/test_unix_pipes.py
--- a/trio/tests/test_unix_pipes.py
+++ b/trio/tests/test_unix_pipes.py
@@ -2,10 +2,11 @@
 import select
 import os
 import tempfile
+import sys
 
 import pytest
 
-from .._core.tests.tutil import gc_collect_harder
+from .._core.tests.tutil import gc_collect_harder, skip_if_fbsd_pipes_broken
 from .. import _core, move_on_after
 from ..testing import wait_all_tasks_blocked, check_one_way_stream
 
@@ -236,6 +237,14 @@ async def patched_wait_writable(*args, **kwargs):
             await r.receive_some(10000)
 
 
+# On FreeBSD, directories are readable, and we haven't found any other trick
+# for making an unreadable fd, so there's no way to run this test. Fortunately
+# the logic this is testing doesn't depend on the platform, so testing on
+# other platforms is probably good enough.
+@pytest.mark.skipif(
+    sys.platform.startswith("freebsd"),
+    reason="no way to make read() return a bizarro error on FreeBSD"
+)
 async def test_bizarro_OSError_from_receive():
     # Make sure that if the read syscall returns some bizarro error, then we
     # get a BrokenResourceError. This is incredibly unlikely; there's almost
@@ -255,5 +264,6 @@ async def test_bizarro_OSError_from_receive():
             os.close(dir_fd)
 
 
+@skip_if_fbsd_pipes_broken
 async def test_pipe_fully():
     await check_one_way_stream(make_pipe, make_clogged_pipe)