I love attempts at visual programming. As a kid I used to pour over the then-fashionable ads for CASE (Compater-Aided Software-Engineering) tools in DDJ and elsewhere and imagined them to do far more than they actually did... Also attempts like Amiga Vision [1]

One of the software engineers I like to go a bit fanboy-ish about is Wouter van Oortmerssen, who I first got familiar with because of Amiga E, but who has a number of interesting language experiments [2], one of which includes a visual language named Aardappel [3] that also used to fascinate me.

There are a number of problems with these that have proven incredibly hard to solve (that this Racket example does tolerably well on, probably because it doesn't go very far):

1. Reproduction. Note how the Amiga Vision example is presented as a video - there is not even a simple way of representing a program in screenshots, like what you see for the examples of Aardappel, which at least has a linear, 2D representation. That made Amiga Vision work as a tool, but totally fail as a language. This is even a problem for more conventional languages on the fringe, like APL, which uses extra symbols that most people won't know how to type. The Racket example does much better in that it can be reproduced in normal text easily.

2. Communication. We talk (and write) about code all the time. Turns out it's really hard to effectively communicate about code if you can't read it out loud easily, or if drawing is necessary to communicate the concepts. Ironically, if you can't read the code out easily, it becomes hard for people to visualise it too, even if the original representation is entirely visual. This example does ok in that respect - communicating a grid is on the easier end of the spectrum.

3. Tools. If it needs special tools for you to be effective, it's a non-starter. This Racket example is right on the fringes of that. You could do it, but it might get tedious to draw without tooling (be it macros or more). On the other hand the "tool" you'd need to be effective is limited enough that you could probably implement it as macros for most decent editors.

I spent years experimenting with ways around these, and the "best" I achieved was a few principles to make it easier to design around those constraints:

A visual language needs a concise, readable textual representation. You need to be able to round-trip between the textual representation and whatever visual representation you prefer. This is a severe limitation - it's easy to create a textual representation (I had prototypes serialising to XML; my excuse is it was at the height of the XML hype train; I'm glad I gave that up), but far easier to make one that is readable enough, as people need to be able to use it as a "fallback" when visual tools are unavailable, or in contexts where they don't work (e.g. imagine trying to read diffs on Github while your new language is fringe enough for Github to have no interest in writing custom code to visualise it; which also brings up the issue of ensuring the language can easily be diffed).

To do that in a way people will be willing to work with, I think you need to specify the language down to how comments "attaches" to
language constructs, because you'll need to be able to "round-trip" comments between a visual and textual representation reliably.

It also needs to be transparent how the visual representation maps to the textual representation in all other aspects, so that you can pick one or the other and switch between the two reasonably seamlessly, so that you are able to edit the code when you do not have access to the visual tool, without surprises. This makes e.g. storing additional information, such as e.g. allowing manual tweaks to visual layout that'd require lots of state in the textual representation that people can't easily visualise very tricky.

Ideally, a visual tool like this will not be language specific (or programming specific) - one of the challenges we face with visual programming, or even languages like APL that uses extra symbols, is that the communications aspect is hard if we can not e.g. quickly outline a piece of code in an e-mail, for example.

While having a purely textual representation would help with that, it's a crutch. To "fix" that, we need better ways of embedding augmented, not-purely-textual content in text without resorting to images. But that in itself is an incredibly hard problem, to the extent that e.g. vector graphics supports in terminals was largely "forgotten" for many years before people started experimenting with it again, and it's still an oddity that you can't depend on being supported.

Note that the one successful example in visually augmenting programming languages over the last 20-30 years, has been a success not by changing the languages, but by working within these constraints and partially extracting visual cues by incremental parsing: syntax highlighting.

I think that is a lesson for visual language experiments - even if you change or design a language with visual programming in mind, it needs to be sort-of like syntax highlighting, in that all the necessary semantic information is there even when tool support is stripped away. We can try to improve the tools, but then we need to lift the entire toolchain starting with basic terminal applications.

[1] https://www.youtube.com/watch?v=u7KIZQzYSls

[2] http://strlen.com/programming-languages/

[3] http://strlen.com/aardappel-language/