One rendering tree
By: Tudor Girba
Tags: Moldable Development, Development Environment, Gtoolkit, Graphical User Interface

There are a few of things to notice here. Each word and each space are elements. The red rectangle is part of the same tree. And, the cursor itself is part of the same tree, as well. As the entire editor is made of elements, the arrangement of these elements is expressed by a layout. A flow layout, in our case.

This organization offers a high degree of flexibility. One immediate application is that of adornments, dynamic artifacts that can appear in the text. In our example, the word “moldable” is colored in blue and uses a monospace font. Nowadays, these are typical abilities in a text editor, and they are supported through text attributes. In our case, the code looks like this:

(text from: 5 to: 12)

attributes: {

BlFontFamilyAttribute named: 'Source Code Pro'.

BlTextForegroundAttribute paint: Color blue }

Less obvious is that our editor treats the red rectangle as a text attribute, too:

(text from: 39 to: 39)

attributes: {

BrTextAdornmentDynamicAttribute new

beAppend;

stencil: [

BlElement new

size: 70 @ 50;

margin: (BlInsets all: 5);

background: Color red ] }

We call these adornments, and they are just a specific kind of text attributes.

Armed with this, we can start playing with the text styler. A styler relies on a little process that goes through the text and augments it with various attributes. As adornments are attributes, the styler can augment the text with arbitrary visual elements, too. Let’s take a look.

Here we see a Coder editor opened on a Pharo method. The interesting thing about this is that all messages sent to self get a little triangle next to them. These triangles are added by the syntax highlighting, dynamically, as you type. When expanded, another editor appears inline showing the corresponding method.

Coder on a method with messages expanded in place.

New kind of programming interfaces

This infrastructure makes possible all sorts of new interfaces.

Let’s start with the unfolding code editor from above. This is interesting in a couple of ways. First, we now get an interface that allows us to read code as a contiguous narrative. This means that we no longer have to choose between short methods and more “readable” code (This debate typically becomes apparent when people familiar with C-like programming languages start learning another language and get confused by the myriad of tiny methods. No more.).

But, it gets even more interesting when we combine this ability with knowledge about specific parts of the system. For example, below you can see a Coder opened on the baseline for Glamorous Toolkit itself. What’s a baseline? It’s a configuration describing what a project is made of and what its dependencies are, along the lines of Maven or npm. The baseline is written through a fluent interface provided by the Metacello library. As with any configuration, it relies on strings for pointing to the contained packages and the dependent projects. In our case, if the dependent project is already available in the image, we show a triangle that, when expanded, reveals the baseline of the respective project. This is a simple thing, but it simplifies creating these baselines significantly without requiring one to learn a new tool.

Coder on the baseline of Glamorous Toolkit showing some of the dependent projects expanded.

The expansion adornments also paved the way to reinventing documentation. For example, below we see a snippet from the Glamorous Toolkit documentation. The document contains text and a graph showing all dependencies of Glamorous Toolkit extracted automatically from the baseline. The more interesting part is that the graph is embedded in the document through a markup instruction. Furthermore, the markup appears only when the cursor gets closer to it. Essentially, we get a new class of editors that is in between editor+view and a WYSWIG (what you see is what you get) interface. We get the power of programmability through markups while still retaining direct manipulation.

Documenter showing a document containing a diagram that is created dynamically based on a markup. The document relies on a markup that appears only when the cursor is nearby.

One more example is an interface for supporting explanations. In this case, the editor on the left has terms that are marked with special attributes through which we get elements that are inserted in the text and that are linked with the markers on the right when hovering over them. For this to work, the explanation word, the target anchor and the line must reside in the same rendering tree.

A live explanation. The terms from the editor on the left are linked to the visual items on the right.

The same mechanism can be used to explain snippets of code. Here, the method on the right famously utilizes the entire syntax of Pharo. On the left, we have an explanation that links to the various parts. Interesting in this case is that the adornments in the code snippet are added through syntax highlighting.

A live explanation of code. The explanation lines provide a natural connection between the explanation and the corresponding code piece. Furthermore, the links change live as the code changes.

The annotated tree

One rendering tree allows us to combine arbitrary elements in a graphical scene. But, there is one more thing. A typical approach to graphical user interfaces is to rely on higher level models that generate the actual rendering tree. Often these higher level models can be complicated and come as a layer on top of the base tree. This then hinders the creation of fluid interfaces because the higher level of abstractions usually can do less than the lower level ones.

We wanted one single space. One rendering tree to rule them all. Now, of course, we also want to be able to compose elements at higher level of abstractions. To accommodate this, we turned the problem around: we made the basic elements annotatable with higher level constructs.

For example, below we see on the left a scene with an inspector embedding a visualization. On the right, we see the rendering tree of that scene. Hovering over elements shows that they are all part of the same tree.

The tree on the right shows how the highlighted element on the left fits into the rendering tree.

The tree on the right also reveals some nodes as labelled. In our case, the labels are phlow and graph . These are annotations. Phlow is the engine for defining inspector views and workflows, while the graph is … well, a graph. So, what can we do with this?

For example, given an element, we can ask it for element graph connectedEdges . graph is a tiny object that knows the element and essentially extends the interface of the element with higher level predicates. Similarly, phlow offers predicates distinguishing whether an element defines things like a view title or a view content.

Following this pattern, we always rely on one single tree and we make it the responsibility of various libraries to enrich that tree. The outcome is that we can combine arbitrary abstractions rather seamlessly. For example, in our case, the root of the visualization is annotated with both graph and phlow it is also the content of the view.

Before we part

In the age of 3D games, AR/VR and smart physical environments, it is perhaps strange to talk about a 2D graphical engine. Still, our audience is fairly rooted in sitting in front of a desktop and working with rather antiquated interfaces. Our audience are programmers, and our goal is to reimagine the development experience.

Indeed, in this day and age, the typical development experience is dominated by text. In the rare cases when text is augmented with visuals, that augmentation happens on the outskirts. That’s because the editor is too special a box. To break this, we set to make the text be made of tiny regular elements. Along the same line, we also wanted visualizations to be first class citizens. So, we had to accommodate those too, and naively perhaps, we formulated the one rendering tree principle that we wanted to stick with at all times.

Of course, having a rendering tree is not new. Of course, there are text editors that are made of nice small elements. Of course, extensive visualization engines do exist. Still, having them all together in one cohesive and reasonably performant infrastructure is new. At least to us.

But, we think the real challenge is only yet to come. We have built a vehicle, but we believe the challenge lies in the uncharted territory.

Just consider this example again. The visualization is not confined to a compartment in a bentobox-like interface. Instead, it is the visualization that defines the frame for the widgets. And while there are visual boundaries around the editors and inspectors, they remain exactly that: visual. The logical connections are not affected and can cross those boundaries to create a seamless scene.

The visualization defines the frame and embeds rich widgets that can be visually and logically connected.

This is not a small change. This is an entirely new kind of interface. And this is but one example. We are still learning how to think about designing interfaces in this world, but the adventure does appear to be exciting.