Text to GIF animation — React Pet Project Devlog
Tags: webdev, javascript, react, showdev
Published: 2020-06-17 18:07:15

I had an idea that it'd be cool to create storyboards using simple text:

![the concept with two frames](https://dev-to-uploads.s3.amazonaws.com/i/03uoorq4n7k5u9x9v0oo.png)

<figcaption>User describes a scene and gets images as an output</figcaption>

In this article, I'll share with you my thought and build process of this project.

And surely I'll show you the final result!

> **Spoiler:** it didn't end as planned. [tl;dr final result](http://framd.cc/)

### 0. Idea

I've started with visualizing the idea. First pen & paper, then [draw.io](draw.io) (you can see one of the first concepts above), then writing a generic overview:

> User defines scenes using simple English:

> `Scene`
> `Ann as woman to the right`
> `Frame`
> `Ann says "Hello!"`

> Scene definitions describe **actors** (decorations) and **frames**
> **Actors** use preset sprites like `tree`, `man`, `woman`
> **Frames** define actions that the actors perform

> Users will be able to share their stories and "like" and modify other's.

With the idea and language more-or-less defined it was time to draft the development plan:

1. [Set up](#1-set-up) — _project preparation_
2. [JSON to images](#2-json-to-images) — _test out if I can create images as I want_
3. [Images to GIF](#3-images-to-gif) — _ensure I can generate gifs on the client, try some libs_
4. [Text to JSON](#4-text-to-json) — _I'll need to create a parser for the language_
5. [Backend](#5-backend) — _needed for login/save/share flows_
6. [Publication](#6-publication)

_**NOTE:** For brevity reasons, I wont mention many dead-end ideas or stupid mistakes: so if you'll get a feeling that everything goes too smooth — thats only because of the editing. Also, I will cut a lot of code and use pseudocode-ish style to shorten the sources. If you'll have any questions — please, don't hesitate to ask!_

Let's go!

## 1. Set up

I'll need a git repository to organize my dev process and a framework to speed it up. I used [create-react-app](https://github.com/facebook/create-react-app) and [git init](https://githowto.com/create_a_project):

```sh
npx create-react-app my-app
cd my-app
# [skipped TypeScript adding process]
git init -y
git commit -m "initial"
npm start
```

**Important thought:** We need to test our ideas fast! And it doesn't really matter which language, framework, or VCS you use, as soon as you're comfortable and productive with it.

## 2. JSON to images

I started with defining a simple JSON to test out if I can render images based on this structure.

The JSON should describe:
- `sprites` — image URLs for our actors and decorations
- `scenes` — should contain and position actors and decorations
- and `frames` — should contain actions, like "Ann moves left"


```js
({
  sprites: { name: 'http://sprite.url' },
  scenes:
    // scene descriptions
    { scene_ONE:
        { entries:
            /* entries with their sprites and position */
            { Ann: { sprite: 'woman'
                   , position: { /* ... */ }
                   }
            }
        },
    },
  frames:
    [ { scene_name: 'scene_ONE'
      , actions: [
          { target: 'Ann'
          , action: 'move'
          , value: {x, y}
          }
        ]
      }
    , // ...other frames
    ]
})
```

For the **actors** I've defined three preset sprites: `tree`, `woman` and `man` and added relevant images to the project.

-

Now for each frame we'll perform all the actions (move and talk)

```js
// for each frame
const computedFrames = frames.map(frame => {
  // clone entries
  const entries = _.merge({}, frame.scene.entries);
  // perform actions on the target entry
  frame.actions.forEach(action => {
    const entry = entries[action.target];

    if (action.type == 'talk') {
      entry.says = action.value;
    }

    if (action.type == 'move') {
      entry.position = action.value;
    }
  });

  return { entries };
});
```
-

And for drawing entry sprites we'll surely use [canvas](https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D):

```js
// draw the entries
const images = computedFrames.map(frame => {
  const canvas = document.create('canvas');
  const ctx = canvas.getContext('2d');
  frame.entries.forEach(entry => {
    ctx.drawImage(entry.sprite); // for sprites
    ctx.fillText(entry.says);    // for speech
  });
  // return rendered frame URL
  return URL.createObjectURL(canvas.toBlob());
})
```

Canvas can export it's contents as a [dataURL](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL) or [a blob](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob) — we'll need this to generate .gif later!

_^ In reality, the code is a bit more asynchronous: `toBlob` is async and all the images should be downloaded before `ctx.drawImage`, I used a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) chain to handle this._

At this point I have proved that the images can be rendered as intended:

![the first result](https://dev-to-uploads.s3.amazonaws.com/i/q0aggtbiwsvjp4t1no54.png)

<figcaption>The First Rendering! Hurray! 🙌</figcaption>

So we can move on:

## 3. Images to GIF

This required some research on available libraries. I've ended up with [gif.js](https://github.com/jnordberg/gif.js). Alas, it hasn't been updated for about a year atm, but it did it's job quite fine ([demo](http://jnordberg.github.io/gif.js/)).

To generate a .gif file — we need to feed every image to the `gif.js` generator and then call `render()` on it:

```js
const gif = new GIF({ /* GIF settings */ });

images.forEach(imgUrl => {
  const img = new Image();
  img.src = imgUrl;
  gif.addFrame(img, { delay: 1000 });
});

gif.on('finished', blob => {
  // Display the blob
  updateGifURL(URL.createObjectURL(blob));
});

gif.render();
```

Awesome, now we can generate and download the .gif:

![gif.js generation](https://dev-to-uploads.s3.amazonaws.com/i/nfe7kqxgm1buwaavpmdi.gif)

<figcaption>Aand action!</figcaption>

## 4. Text to JSON

I wanted users to type in commands in simple English. And this was the hard part for me, as I didn't even know where to start:

- create my own parser?
- `input.split(/\n/)` and then use regexes?
- use some English grammar parser?

Luckily, after searching around I came across this article ["Writing a DSL parser using PegJS"](https://dev.to/barryosull/writing-a-dsl-parser-using-pegjs--4igo) that introduced me to PEG.js (@barryosull, thank you).

[PEG.js](https://pegjs.org/) is a simple-to-use parser builder:

1. you define your language using regex-like rules  
2. it generates a `.js` file with your personal fresh new parser
3. you plug this parser in and run it against your text

For example, here is an excerpt from my rules that parses `Scene`s:

![PEGjs excerpt](https://dev-to-uploads.s3.amazonaws.com/i/vli8b9yyznh4gqgg3di3.png)

These rules will parse this text:

> `Scene`
> `with Tree  as tree at { y: 160 scale: 1.5 }`

to this JSON:

```json
{
    "type": "scene",
    "values": [
      {
        "type": "object",
        "objectName": "Tree",
        "sprite": "tree",
        "state": {
            "y": 160,
            "scale": 1.5
        }
      }
    ]
}
```

In a couple of hours playing with PEG.js [online version](https://pegjs.org/online) I ended up with a language and output structure I was fine to work with.

And after plugging it into the app I got this:

![PEGjs parsing screenshot](https://dev-to-uploads.s3.amazonaws.com/i/054s6aik90ibhk4crtvh.png)

<figcaption>Look, they talk and listen</figcaption>

**SIDE NOTE 1:** at this point, I dropped the idea of fuzzy positioning `Ann to the right` and updated PEG.js to define a js-like object notation: `Ann at { x: 100 y: 100 }`.

**SIDE NOTE 2:** also, I couldn't keep re-generating GIF on every text update. It was too heavy. 100ms-UI-thread-blocking-on-every-keystroke heavy.

**RxJS** 😍 came to the rescue! [debounce](https://thinkrx.io/rxjs/debounceTime-vs-throttleTime-vs-auditTime-vs-sampleTime/) for input text updates and a simple [timer](https://thinkrx.io/rxjs/timer/), mapped to frame switch `imgRef.current.src = next_frame` to _imitate_ the animation.

Actual .gif will be generated only when user hits the "download" button!

## 5. Backend

This pet-project already took me a weekend of development, so I had to drop all backend-related tasks and stick with static web-app for now.

## 6. Publication

I used [GitHub Pages](https://pages.github.com/) feature to deploy and share the project.

GitHub Pages will serve your website under their domain `http://username.github.io/repository`. Since I might be adding a backend later — I needed to buy a domain so that all links I share now would still be valid in the future.

Picking names is always hard for me. After an hour struggling I ended up with:

![result](https://dev-to-uploads.s3.amazonaws.com/i/2hq6axhixjqg4t0az5ry.png)

Go give it a try [framd.cc](framd.cc)! 🙂

## Outro

**Plot twist:** After deploying and sharing project with friends — I've found that I'm short on sprites! **Turns out people can't tell a lot of stories using just man, woman, and a tree image**. So I decided to use emoji as sprites 👻 . And now you have a ton of these 🌳👩‍🚀🌍 to [tell your story right](framd.cc)!

## The end

That's it! Thank you for reading this! 🙏

Have any questions? Post em in the comments section, I'll be glad to answer!

If you enjoyed reading — please, consider giving this article and this tweet a ❤️ 

{% twitter 1273316541724049409 %}

**It helps a lot!**

**Thanks!**

**P.S:** some gif examples:

![hearts](https://media.giphy.com/media/d6Jy0VvjwpIbPY2b3z/giphy.gif) ![hatch](https://media.giphy.com/media/XdD6054HARvl0Jlvij/source.gif) ![what](https://media.giphy.com/media/L1bmLK6i1Hi1TDhDGk/source.gif) ![monkeys](https://media.giphy.com/media/l094JtCRTB6qAFdcBp/giphy.gif) ![sixer](https://media.giphy.com/media/ZBKzojnJe4WfIAaKCI/source.gif)