Graphology: Writing a graph of nodes UI in HTML5 (Part 1)
Tags: html, ui, javascript
Published: 2017-04-22 10:01:25

---
title: "Graphology: Writing a graph of nodes UI in HTML5 (Part 1)"
published: true
description: In a series of experiments, I will write a UI for interacting with complex graphs of nodes, each of which may contain some controls of its own.
tags: html5, ui, javascript
cover_image: https://thepracticaldev.s3.amazonaws.com/i/wcxadpczclbbkfsuaqag.png
---

I like Blender's compositor nodes.

![Blender compositor screen from some Reddit post](http://i.imgur.com/LBSEWh4.png)

It doesn't mean I know how to use them well, but I like the UI and I think it works very well in that context.

I also kinda like the JACK Patchbay, because it shows you exactly where everything is connected - and it uses a similar interface.

![A Patchbay running inside of Carla, a music production app](http://kxstudio.linuxaudio.org/screenshots/carla-patchbay.png)

We will not talk about the ALSA Modular Synthesiser here.

I like these interfaces because they show you how everything is connected in a complex graph, where you can zoom in on a particular part and change some settings of a single node, or zoom out and get a good feel of how the data (be it pixels, vector data, sound, whatever) flows through the graph.

So I want to write a general-purpose graph interface for working with graphs of complex nodes. And I want to use a data-driven approach so that, if you want, node UIs will be generated automatically.

## So far

A scattershot experimental approach. Here is the sort of UI I'm after. It's pretty horrible at responding to screen sizes.

![A screenshot of the nodes UI example showing a two-node graph](https://thepracticaldev.s3.amazonaws.com/i/sk7nfzihx4dju0igsa1c.png)

[Nodes UI example](https://codepen.io/FLamparski/pen/LyErym)

Here's a simple demo of one of the interactions I'm after: [dragging around Bezier curve connectors](https://codepen.io/FLamparski/pen/JNoZEM).

Note that this may very well be a horrible and hackish approach. It uses _SVG_ to render the connector, and changes the path live as you drag its free end with the mouse.

## Existing software

**d3** - Seems like it might help in some respects but I couldn't find a demo that captures what I want. Also I want this UI to be compatible with React (more or less) which d3 isn't.

**jsPlumb** - [This demo](https://jsplumbtoolkit.com/demos/toolkit/groups/index.html) captures an _essence_ of what I want to do. It's ugly as a naked mole rat, and the library itself is priced in $k's per year. I want my UI library to be open source (with reasonable commercial licences should this concept catch on).

## Basic data model

I am assuming the following things for the following example:

1. Nodes are representations of functions
2. A node can have many inputs and outputs

```javascript
import {Node, Input, Output} from 'nodes-ui';
import {Colour} from '../image/Colour';

export const ColourMixer = Node({
  type: 'ColourMixer',
  name: 'Colour Mixer',
  inputs: {
    colour1: { name: 'Colour 1', type: Colour },
    colour2: { name: 'Colour 2', type: Colour }
  },
  settings: {
    mixType: { name: 'Mix function', type: String }
  },
  outputs: {
    output: { name: 'Output', type: Colour }
  }
});
```

That would be an individual node's definition. Perhaps a special case might be necessary for nodes that provide external input ("sources") or describe external outputs ("sinks") in the graph. The UI will take in those node definitions do some magic to construct graphical representations of those nodes, letting you add and remove them and link them together into graphs. Finally, it will give you back a graph of objects like this (where the `#'id'` notation should be thought of as "a reference to a Node object with this ID):

```javascript
{
  id: 'ColourMixer-35',
  type: 'ColourMixer',
  settings: {
    mixType: 'multiply'
  },
  inputs: {
    colour1: {from: #'Image-24', output: 'colourData'},
    colour2: {from: #'Colour-5', output: 'colour'}
  },
  outputs: {
    output: {to: #'Display-1', input: 'colourData'}
  }
}
```

(And I guess at the top level, you'd have to have an object like `{sources: [...], sinks: [...]}`)

From there, you'd use this graph representation to construct your internal computation graph or whatever it is you want, and then run some computations on that structure.

**What do you think of this idea?** Am I reinventing the wheel? Is there a library that can already do this? Or do you have a suggestion for improvements? Let me know in the comments!