Well put! Text is graphics, but technology has historically limited how it looks and what you can do with it. The idea of text with control characters like "carriage return" and "newline", fixed width text, limited length lines, and text editors that wrap words when the cursor gets to the edge of the screen, and terminals with multiple lines that scroll, all incrementally evolved out of emulating punched card machines. And now we emulate VT100s emulating punched card machines. But it doesn't have to be that way.

Here's a great piece of exploratory research that imagines how things could be different, by an amazing visionary visual thinker and artist, Scott Kim:

"Viewpoint" is part of Scott Kim's PhD dissertation that he did at Stanford in 1988 (with Donald Knuth as principle advisor). It was programmed in Cedar, and ran on a color Dorado workstation at Xerox PARC.

https://www.youtube.com/watch?v=9G0r7jL3xl8

>Demo and explanation of Viewpoint, a computer system that imagines how computers might be different had they been designed by visual thinkers instead of mathematicians. Caution: this is basic research, not a proposal for a practical piece of software. Part of my PhD Dissertation at Stanford University in 1988.

At 12:22, he starts with a blank screen, and performs a "visual boot", to use Viewpoint to build itself.

http://www.scottkim.com/

https://en.wikipedia.org/wiki/Scott_Kim

https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint...

This is a visualraccoon Perspective on Scott Kim‘s very graphic and revolutionary exploration, Viewpoint: Toward a Computer for Visual Thinker.

What would it be like to go back to visual first principles and take a fresh look at graphic user interfaces?

The Viewpoint Thesis is that a small number of pixel manipulation primitives can be defined such that if they are bound to keyboard and mouse actions it is then possible to build a simple text-graphic editor by drawing it, and that that editor can be used to draw-build itself.*

The Viewpoint Thesis & Editor is part of a larger project founded on the hypothesis that:

“Only by treating the screen itself as a first class citizen will we be able to build computers that are truly for visual thinkers.” Scott, 1987.

This project includes building visual programming languages for such thinkers.

Viewpoint: Toward a Computer for Visual Thinkers

http://www.scottkim.com.previewc40.carrierzone.com/viewpoint...

http://www.scottkim.com.previewc40.carrierzone.com/viewpoint...

When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay's vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers.

At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually.

It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had.

So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand.

My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” — questions that often get passed over in computer science (any definition of “programming” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas.

I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Please email me your thoughts. I'd love to hear them.