Change:
virtualisation.qemu.options = [ "-vga none -device virtio-gpu-pci" ]; }; <del> enableOCR = true; </del> testScript = { nodes, ... }: '' import shlex <ins> import json q = shlex.quote NODE_GROUPS = ["nodes", "floating_nodes"] def swaymsg(command: str = "", succeed=True, type="command"): assert command != "" or type != "command", "Must specify command or type" shell = q(f"swaymsg -t {q(type)} -- {q(command)}") with machine.nested( f"sending swaymsg {shell!r}" + " (allowed to fail)" * (not succeed) ): ret = (machine.succeed if succeed else machine.execute)( f"su - alice -c {shell}" ) # execute also returns a status code, but disregard. if not succeed: _, ret = ret if not succeed and not ret: return None parsed = json.loads(ret) return parsed </ins> <del> def swaymsg(command: str, succeed=True): with machine.nested(f"sending swaymsg {command!r}" + " (allowed to fail)" * (not succeed)): (machine.succeed if succeed else machine.execute)( f"su - alice -c {shlex.quote('swaymsg -- ' + command)}" ) </del> <ins> def walk(tree): yield tree for group in NODE_GROUPS: for node in tree.get(group, []): yield from walk(node) def wait_for_window(pattern): def func(last_chance): nodes = (node["name"] for node in walk(swaymsg(type="get_tree"))) if last_chance: nodes = list(nodes) machine.log(f"Last call! Current list of windows: {nodes}") return any(pattern in name for name in nodes) retry(func) </ins> start_all() machine.wait_for_unit("multi-user.target")

Discussion:
the sway test is flaky, both because ocr itself seems to flaky and because ocr seems to be very slow. Steps to reproduce the behavior: a couple times, we haven't gotten it to work even once this seems to be blocking nixos-unstable channel updates. there are a number of timeouts in the hydra build logs that we can very clearly attribute to either ocr not working, or ocr timing out because it gets only one shot at finding the gpg prompt (and if that fails, loops forever on the empty desktop).
I found it failing quite reliably, exactly since merging the .
scrolling throug hydra logs it's been flaky before too though, something is definitely up. there's failures in early may that look exactly like what we see now
Where do you see this? Looking through I only see successful builds (apart from the cancelled builds and dependency failures). I also don't remember it ever failing when working on it. So this definitely seems to be a regression from staging-next (either it just causes issues with (some of) the OCR tests or it can even affect users - in which case it would be very good that the test is failing). Unfortunately I don't really have time to look into it right now (maybe Thursday or Saturday). It could be due to a font rendering regression. Hm, that's strange - the GnuPG pinentry-gnome3 passphrase prompt shouldn't disappear automatically / after a short timeout (if that's the case than we definitely need to change that part).
Both and failed consistently at this point: So it looks like either doesn't start when using the Wayland backend or the font rendering is broken then (with the X11 backend (XWayland) the test succeeds).
Some of the “cancelled” builds are cancelled due to it timing-out, e.g. and ; the reason there may be unrelated, though. Indeed most appear to be non-timeout cancellations.
run locally here the things the tests wants to happen did always happen, but OCR was incredibly flaky and either did not detect them at the first try, or not at all. this is especially bad for pinentry because it closes after 10 seconds for reasons we have not dug into.

it looks like this is squarely a failure of OCR (and the way it's used here does seem at least somewhat suspect, maybe using other tooling to list windows would be more appropriate? would almost certainly not take three minutes per attempt to find a window)
Ah, right, thanks for the examples :) Indeed, that seems to be to the gnome3-pinentry timeout that I wasn't aware of (so that part would be fixable). Weird, that shouldn't happen. OCR is slow but should be reliable. Not sure if this could be setup dependant. On Hydra it doesn't seem to be an issue. That is indeed bad. Luckily already found a solution for the timeout in . I intentionally chose OCR to ensure that the (font) rendering works but it isn't required for ensuring that the windows are there. The slowness of OCR is certainly a drawback though.