I quite strongly disagree that Scott Kim's PhD thesis on Viewpoint isn't relevant to a discussion of visual programming languages, and I'll be happy to explain why.

But I don't understand why you have wasted so much time and effort trying to convince me not to discuss it, when you could have simply not said anything at all, since you don't have anything useful to contribute.

If your problem is that there is too much irrelevant noise in this discussion, you have nobody to blame but yourself, because you have added absolutely nothing useful or positive or interesting, just a lot of useless whining. You've already said you don't care FOUR times, so stop denying that you care so frequently, or simply stop posting.

People who don't have anything useful to contribute, or refuse to contribute anything useful that they claim to know but want to keep secret because they don't trust anyone not to steal their idea, shouldn't mention their secrets in the first place, and shouldn't attack other people for contributing things to the discussion.

Stop complaining, and start contributing.

Now back to the point:

I had a discussion in 1999 with Jaron Lanier about the 3D tree node data structure (Swivel3D trees) and plug-in COM data structures in Body Electric / Bounce, and he raised some interesting points about visual thinking and explicit visual representation of data, which parallel what Scott was exploring in his Viewpoint thesis.

Jaron, who founded VPL Research, developed and used Body Electric extensively, programming real time virtual reality simulations and musical instruments by integrating data gloves, body suits, eyephones, two separate SGI workstations to render for two eyes, 3d input trackers, music synthesizers, convolvotrons, and other i/o devices.

https://en.wikipedia.org/wiki/Jaron_Lanier

https://en.wikipedia.org/wiki/VPL_Research

In case you're not familiar with his work, here is a classic 1986 interview with Jaron from the book "Programmers at Work" (which also interviews Scott):

>Today I posted the Jaron Lanier interview from his early days as a young programmer in California. Back then, this free-spirited guy was touting Virtual Reality to disbelieving stares. Jaron has become a great spokesman and sage for the industry, questioning where we are going and where we have come from. He is an author, computer scientist, and gadfly of the industry. You can read all about his recent work here.

https://programmersatwork.wordpress.com/jaron-lanier-1986/

>INTERVIEWER: What are you doing with programming languages now?

>LANIER: Well, basically, I’m working on a programming language that’s much easier to use.

>INTERVIEWER: Easier because it uses symbols and graphics?

>LANIER: It needs text, too. It’s not exclusively graphics. With a regular language, you tell the computer what to do and it does it. On the surface, that sounds perfectly reasonable. But in order to write instructions (programs) for the computer, you have to simulate in your head an enormous, elaborate structure. Anytime there’s a flaw in this great mental simulation, it turns into a bug in the program. It’s hard for people to simulate that enormous structure in their heads. Now, what I am doing is building very visual, concrete models of what goes on inside the computer. In this way, you can see the program while you’re creating it. You can mold it directly and alter it when you want. You will no longer have to simulate the program in your head.

Jaron designed the musical visual program (which Scott Kim cited in his thesis) on the cover of the September 1984 Scientific American on Computer Software (a wonderful issue, with many articles about programming languages and software by some amazing people).

https://www.scientificamerican.com/magazine/sa/1984/09-01/

To draw a parallel, pixels are to Viewpoint as the Swivel3D scene graph is to Body Electric, and they both share the ideal that "the virtual world and the knowledge base were the same thing" and "it's user interface all the way to the bottom":

"I had always thought the swivel tree was ridiculous, of course, but on
the other hand I liked the idea that the virtual world and the
knowledge base were the same thing- that unity encourages the
visibility and grabbability of the underlying concepts. I think the
brain works that way- there isn't some barrier behind which everything
gets abstract- instead, it's user interface all the way to the bottom!
What I think would be the coolest long-term destination of BE would be
extending the scenegraph so that it was as powerful a knowledge base
as you'd want..." -Jaron Lanier

https://macintoshgarden.org/apps/swivel-3d

https://www.donhopkins.com/home/archive/visual-programming/b...

At 12:31 AM -0400 7/8/99, Hopkins, Don wrote:

Hi, Jaron. We've met briefly once or twice - I'm a friend of David
Levitt's.

What's this about Sun acquiring the rights to the VPL patents, and
Body Electric?

https://web.archive.org/web/20051217153424/http://www.advanc...

Does Sun actually have the Body Electric source code? What version do
they have?

Does anybody at Sun even know how to compile a Mac program?

Last time I worked there (admittedly a long time ago), they were too
embarrassed to allow Macs in the building (which might make somebody
realize how bad Unix sucks in comparison).

Has anyone tried to rewrite it in Java?

When I was working with David at Levity and Interval, I totally
overhauled the source code to Bounce: porting it to the latest version
of CodeWarrior and the PowerPC, cleaning up C code translated from
Pascal, that no human had ever touched before, and just generally
re-indenting and adding white space so it was pretty to look at.

Then I implemented a new interface for plugging in DM's, based on
ActiveX (yes, ActiveX runs on the Mac). I added a new data type that
you can flow along blue wires: a COM object (aka a plug-in ActiveX
object in a shared library).

Then we made plug-ins modules that produced and consumed the new
plug-in data types, like strings and polymorphic dictionaries. With
these new data types, we were able to model very complex simulations
as nested trees of dictionaries, strings and numbers, treat
dictionaries as high level objects, pass them all around on wires,
reading and modifying them at will.

The thing that the original Body Electric was missing is a way to
dynamically model structured data like Lisp s-expressions and
association lists, which is now possible in Bounce, using
dictionaries. (The swivel3d trees just don't cut it for representing
"knowledge".)

Bounce still had the "M4" Director player rendering engine, but it
didn't do everything we needed, so I implemented my own graphics
library for drawing sprites, playing sounds, and stuff like that. We
used it to implement a simulation of Rush Limbaugh and Jesse Jackson
watching TV and arguing over the closed captioning stream.

A big fat housefly would skitter around the screen, land on their
faces, and tickle them into waving their hands around and
pontificating.

I just got a 400 mhz G3 powerbook, and fired up a 2 year old copy of
Bounce and the crazy Limbaugh/Jackson demo, and it still works,
actually like a bat out of hell! I ran into David and John Szinger
(another Bounce programmer from Interval) a few days ago at a 4th of
July party, and they really got a kick out of seeing the old demo that
we worked together on, still running!

I live in Oakland just south of Berkeley. Drop me a line if you're in
the bay area, and would like to see what Bounce has evolved into.

-Don

From: Jaron Lanier
To: Don Hopkins
Sent: Thursday, July 08, 1999 6:13 AM
Subject: Re: Body Electric lives?

Hey there, and thanks for writing!

Yup, Sun owns VPL and Body Electric, and my guess is that if anyone
looked very closely it would turn out Interval doesn't have rights to
Bounce. But no one is likely to look very closely, so let's forget
about that. (I don't think Sun is aware of Bounce or the work at
Interval.)

I had never asked what was done with Bounce at Interval- it's
fascinating to hear what you were up to. I live in NYC for the most
part, but I'd love to see it sometime when I'm in the Bay Area. Also
Chuck Blanchard lives in SF and you and he might want to trade demos
sometime.

There IS a community of Body Electric users. It is STILL building the
most interactive 3D virtual worlds of any tool (though Alice, from
Carnegie Mellon, is the other hot contender). That's SHAMEFUL! While
BE sucks in every other way, all the more recent vr design tools,
especially the vrml ones, simply avoid the problem of deep
interactivity. How could the community be so whimpy, at this late
date?

On the Body Electric side of things, there has also been some work
updating to CodeWarrior/PowerPC (shame we didn't share that work!), as
well as support for Quickdraw3D, OpenGL on the Mac, OMS, and some
other standards.

There have been some changes to the interface, but not as ambitious as
yours. The neatest thing is a debugger/tracer tool that is a pleasure
to use. Since BE is used mostly in 3D domain, there are also some
tools dealing with textures, lights, etc.

I had always thought the swivel tree was ridiculous, of course, but on
the other hand I liked the idea that the virtual world and the
knowledge base were the same thing- that unity encourages the
visibility and grabbability of the underlying concepts. I think the
brain works that way- there isn't some barrier behind which everything
gets abstract- instead, it's user interface all the way to the bottom!
What I think would be the coolest long-term destination of BE would be
extending the scenegraph so that it was as powerful a knowledge base
as you'd want...

I'd LOVE to see a fancy BE release, in JAVA, or at least spitting JAVA
out. The question, of course, is where the money would come from.
I've tried to talk Sun into an open source release so the community
could hack on it voluntarily (the source was included in the patents
that were granted, so it's actually already released by the US
government anyway, though only on paper). Unfortunately, Sun doesn't
want to do that. What they've said instead is that I can choose up to
six sites with free hacking privileges at a given time. Bizarre!
With so few sites, I think there'd need to be money to make sure the
sites stayed focused on it... You'd have thought Sun would know
better by now.

The body electric community is surprisingly NOT entertainment
oriented, though I and a few others still use it that way. There are
people hidden away who are still using it for ergonomic simulations,
simple surgical planners (because the fancy systems are too rigid to
model some situations), cognitive test rigs, and some work with kids.

You should know there was some tension about BE/Bounce at one point.
The problem was that Chuck Blanchard wasn't credited as the lead
designer/programmer of BE/Bounce when David brought the program to
Interval. Chuck's name was reduced in stature in the "about" and he
was not mentioned in some important semi-public demos at Interval. He
was also offered a pseudo-position to help with Bounce at Interval,
but without health benefits - and Chuck has MS and absolutely NEEDS
health insurance. I at one point yelled at the guy who runs Interval
(forgot his name..) about their treatment of Chuck- and David is
STILL mad at me for making a fuss about it. But I felt I had to.

So that's the scoop! Both sides of the mystery revealed!

All the best,

Jaron

Jaron on the web:

https://web.archive.org/web/20050212085044/http://www.advanc...