Swingland: Recreating Java Swing for Wayland
Tags: java, opensource, showdev
Published: 2023-05-12 23:24:42

---
title: Swingland: Recreating Java Swing for Wayland
published: true
description: A Java GUI library for Wayland that looks like Swing
tags: #java #opensource #showdev
cover_image: https://www.ashbysoft.com/articles/swingland/swingland-lives.png
# Use a ratio of 100:42 for best results.
---
(also published on [my blog](https://www.ashbysoft.com/articles/swingland/))

Irritation-driven-development
=============================

Back in the past [I wrote](https://git.sr.ht/~phlash/java-sdr) a couple of [Java apps](https://git.sr.ht/~phlash/powermonitor). These were written using the [Java Swing](https://docs.oracle.com/javase/tutorial/uiswing/) framework, and have worked nicely on both Windows and Linux X11 desktops for a long time.

More recently I switched away from X11 & [Budgie](https://blog.buddiesofbudgie.org/) to pure [Wayland](https://wayland.freedesktop.org/) for my desktop on the assumption that it's over 10 years old now, and is the default technology underlying current Gnome and KDE desktops.. everything will be fine right? Kind of..

It turns out Java GUI technology has been somewhat abandoned by Oracle, in particular when running Swing (or JavaFX) applications on Wayland, Java relies on the `XWayland` adpater to bodge X11 API calls back to Wayland proper. This is not great (read: doesn't work out of the box), so I wondered if someone had implemented a native Wayland backend for Swing - [and they have](https://wiki.openjdk.org/display/wakefield/OpenJDK+Project+Wakefield+-+Wayland+desktop+support+for+JDK+on+Linux), [multiple times](https://github.com/CaciocavalloSilano/caciocavallo) - super, but wait.. Oracle have not merged either project into mainline Openjdk, and indeed the only way to make either of them work is by [hideous bodging](https://github.com/CaciocavalloSilano/caciocavallo/blob/master/cacio-tta/src/main/java/com/github/caciocavallosilano/cacio/ctc/junit/CacioExtension.java), which also puts paid to any thoughts of writing my own backend, as the hacks required to load it at run time (read: in every application) are not sustainable. Grrr! So what to do?

I can of course switch back to an X11 desktop (ie: Budgie), but __this irritates me__, plus I have [already ported OpenSceneGraph + Flightgear to Wayland](https://github.com/phlash/OpenSceneGraph/tree/wayland-backend), it wasn't that difficult :)

Swing, like most medium-large frameworks has documented public API and internal implementations, so I _could_ choose to re-write Swing, retaining the public API (thus existing applications can re-compile against my library) but _rewrite Swing internals_ from the ground up on top of Wayland APIs. Sounds sensible right? However Swing (and the earlier AWT it relies on) have [several hundred documented classes](https://docs.oracle.com/en/java/javase/17/docs/api/java.desktop/module-summary.html), with many more undocumented internal classes and a fair amount of native code (in C) to connect to X11, Win32 or Cocoa (macOS), plus there are no Java bindings to Wayland, only the `libwayland` C API. How brave do I feel?

Looking at my existing apps (see above), I have only used a small number of Swing features, so I _should_ be able to [create a spike](https://en.wikipedia.org/wiki/Spike_(software_development)) that supports just the features I need to prove my proposal.. still seems pretty mad, but I have time (the glory of retirement!), so let's go!

Wayland API in (mostly) Java
============================

Given I _will_ be using most of the Wayland protocol to achieve anything (it's minimal) and [Wayland is well specified](https://wayland.app/protocols/) then the 'start at the bottom and build up' design pattern fits. Wayland has a `wire protocol` based on a Unix socket, and usefully Java has supported Unix sockets since release 16, so I can write everything in Java to (de)serialise messages. This gets me going quickly, providing positive feedback that I'm on the right track..

Unfortunately Wayland also uses two Unix features that Java doesn't have: SYS V shared memory buffers; file descriptor passing over Unix sockets. I then discover a bug (or misunderstand the documentation!)..

### Shared memory solution

Helpfully [Java Native Interface (JNI)](https://docs.oracle.com/en/java/javase/17/docs/specs/jni/) _does_ allow memory buffers to be passed from native code back to Java, and vice versa using `DirectByteBuffer` objects - so I only have to write enough JNI/C code to allocate and destroy shared memory, which I can then efficiently access from the Java code.

### Passing file descriptors

This is a less obvious solution, since Java does _not_ provide public access to the native file descritor(s) for any operating system APIs, _however_ the source code is available identifying where a socket stores this native value (which appears to be stable over many releases), and from JNI/C I can easily access the required object field without the security challenges of doing so in managed Java code. I'll live with this slightly hacky solution for now (until Java has accessors for native descriptors, it's been proposed a few times!)

### Checking for input

Wayland operates _asynchronously_ between client and server, there are only a couple of occasions when a client needs to _wait for input_ and we can simply _block in a read call_, at other times we need to _poll for input_ using a check for data being available to read. This _should work_ via the `java.nio.channels.Selector` mechanism, but sadly it does not. Thus I extend my JNI/C a little to provide a working data availability check, using the above hack to obtain the native file descriptor and calling `ioctl(FIONREAD)`, which does work!

## Testing ensues..

As soon as I have enough of the protocol implemented to connect and display something, then testing ensues - gotta see something on screen :) The [test program](https://git.sr.ht/~phlash/swingland/tree/trunk/item/src/com/ashbysoft/test/Wayland.java) will now evolve along with the protocol components as I add features. Oh, and it works nicely - this is 11 days in!

![random noise test pattern](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/43660me13km7ztdhl48m.png)


Swing API in many steps
=======================

With the underlying Wayland protocol up and running, I move on to a 'baby steps' exploration of the Swing API, working down from one of my simple applications (the powermonitor) and creating 'just enough code to pass the tests'. This takes considerably longer than I hoped, as Swing is rather tightly coupled, and small refactorings (such as hoisting state to a super class) result in a lot of changes to coupled code - bah!

Along side the real application, I create a [Swing test application](https://git.sr.ht/~phlash/swingland/tree/trunk/item/src/com/ashbysoft/test/Swingland.java) which provides scope for short cuts and internal testing during development without hacking my real app about (always good to have something you can run after each change!)

After another 5 days work, I get to see the first Swing app run, with almost no useful features, but API compatible with the real thing. Neat!

## Features, then more features

So from here I start picking off features to add and working through the stack to 'make it so' for each: first up a single 8x16 `MONOSPACED` font that I ~~steal~~ borrow from [The Ultimate Old Skool PC Font Pack](https://int10h.org/oldschool-pc-fonts/readme/), the original IBM VGA text mode font. Retro computing FTW.

This is followed up with input handling: first keyboard then mouse, both rather buggy; then more high level components (`Dialog`, `JLabel`, `JButton`) over the next few days, followed by improvements to `Graphics` primitives.. and lots of debugging, particularly around the interaction with my tiling window manager (`sway`) and how it chooses to position new windows.

Cursor support bogs me down for a bit, especially the interaction between an asynchronous protocol, and the need to change the cursor on demand while the pointer is within a single Wayland window, also the increasingly complex logic required to flow input events through the component stack, which I refactor a couple of times.

## Help arrives!

About a month into the project, my eldest son arrives for a family visit, and gets interested in the madness.. he pulls the repository, builds the code and it runs first time, on his Pinebook, which is both an ARM (not x86) and big-endian by default - I'm almost impressed this just works!

He also find and fixes a couple of protocol bugs for me - thanks Martin!

## Dogfooding time

At this point I decide to write an app that both uses Swingland and supports it, a font editor for the custom file format I have created to store simple bitmap fonts - this allows me to create a proper cursor font without hand-editing in a hex editor (which I have been doing until now!)

The process of creating this real application finds more bugs (of course!)..

## Image support

The next feature is the `ImageIO` library that allows bitmap images to be loaded. Because I have contributed to it and I think it's cool, I start with the [Quite OK Image format](https://qoiformat.org/), then [Portable Network Graphics](https://en.wikipedia.org/wiki/PNG) as most images will be in PNGs. Transparency and alpha blending follow as both image formats support alpha channels.

## Window manager interaction

Switching an application to full screen (and back) under program control is explicitly supported in Swing, and useful for games or similar, so this goes in next, and requires yet another refactor of the way Swing interacts with Wayland

## Menus!

At this point we're approaching the 80% useful mark of the [Pareto Principle](https://en.wikipedia.org/wiki/Pareto_principle), Swingland is looking like an actually useful libray (tm), although it really needs application menus.. cue another round of refactoring, popup window handling and event processing re-design, and we have it! Looking good:

![Swingland lives!](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gvtbomujm90mqe32immk.png)

## Getting braver with testing

Having exhausted my own applications as test beds, I decide to move on to the Oracle-provided Swing demo apps, in particular the `TopLevelDemo` and `ButtonDemo` ones that should now work with a simple change of imports and a re-compile.. well almost ;) More debugging and fiddling with the detail of button processing ensues.

## More fonts..

The latest feature to go in is support for X11/PCF font files, which opens up a world of reasonable fonts for text. I had looked at TrueType (TTF) fonts earlier and decided that they were far too much work for now (a whole virtual machine and binary executable format to just fiddle with point positions during layout.. err nope!).

Ongoing development
===================

I hope to maintain Swingland as an actually useful library for some time, and would particularly welcome other contributors who need other Swing features that they could develop. Come get some [hot source](https://git.sr.ht/~phlash/swingland)