Perhaps most of the graph code can be moved outside the `BUILD_X11` conditionals?