Editing Videos Using ffmpeg + TypeScript and Deno.
Tags: deno, typescript
Published: 2023-12-07 17:31:01

> **TLDR**: I wrote a module using typescript / deno that wraps ffmpeg and offers a better developer experience when editing a bunch of video clips together 👉 [deno/x/movie](https://deno.land/x/movie@1.0.0).

I've been recording small clips of video on my old "retro" 🤣 "vintage" 😭 Nikon point-and-shoot digital camera. I'm trying to think more visually and thinking about using video as a medium to explore concepts I'm experiencing in life, and evoke a feeling. 

I kinda hate modern video editing software, mainly using iMovie. I briefly looked into other open-source alternatives to iMovie, but nothing stood out. Then I thought about `ffmpeg`, a command-line tool I've used once or twice in the past to convert video or audio to another format. 

I started a deep-dive in learning `ffmpeg`. How to do things like remove sound entirely from a clip. How to manipulate saturation, gamma, and contrast. How to raise the volume of a clip. All of these are done within `ffmpeg` as "filters". Given that it's a command-line tool every character matters and a lot of information is condensed into a short command. The syntax is a bit hard to understand. I found that breaking  the command down using new lines and a `\` was the only way to really read it. 

## Brief Explainer

Here's a simple script that merges two videos:

```bash
ffmpeg \
-i ./example/originals/DSCN3700.AVI \
-i ./example/originals/DSCN3701.AVI \
-filter_complex "\
[0:v:0][0:a:0]\
[1:v:0][1:a:0]\
concat=n=2:v=1:a=1[outv][outa]" \
-map "[outv]" -map "[outa]" \
./example/output.mp4 -y
```

Here's a simple breakdown of this command:

* `-i` are inputs 0 and 1
* `0:v:0` refers to input 0 and video feed 0
* `0:a:1` refers to input 0 and audio track 0

This is because files can have different video and audio tracks, think about the feature of supporting a video with multiple language tracks. Track 0 may be English, track 1 may be Japanese.

This is a simple example, a more complex example would look like this:

```bash
ffmpeg \
-i ./example/originals/DSCN3700.AVI \
-i ./example/originals/DSCN3701.AVI \
-filter_complex "\
[0:a:0]volume=3,atrim=start=4:duration=4,asetpts=PTS-STARTPTS[audio0];\
[0:v:0]fps=25,crop=480:480:0:0,trim=start=4:duration=4,setpts=PTS-STARTPTS[video0];\
[1:a:0]volume=4,atrim=end=17:start=15,asetpts=PTS-STARTPTS[audio1];\
[1:v:0]fps=25,crop=480:480:0:0,trim=end=17:start=15,setpts=PTS-STARTPTS[video1];\
[video0][audio0]\
[video1][audio1]\
concat=n=2:v=1:a=1[outv][outa]" \
-map "[outv]" -map "[outa]" \
./example/output.mp4 -y
```

* This does the same for imports
* This adds in the notion of a "filter" assigned from a track to a "variable"

```bash
[0:a:0]volume=3,atrim=start=4:duration=4,asetpts=PTS-STARTPTS[audio0];\
  ^ the track -----------[ the filter(s) ]-- the new variable    ^
```

Each of these definitions are followed by a `;`

Then we have the concatenation definition after the last `;`:

```bash
[video0][audio0]\
```

This specifies this audio video pair from the definitions followed by the next and so on and so on.

So I view the `-filter_complex` value as this:

```bash
{all the filter transformations, declaring a new variable };
{the concatenations, paired audio + video}
```

## I built a wrapper

Early when learning the filters I wanted a better way so I created some glue-code to wrap `ffmpeg` in a way that was easier to move clips around and be more flexible. 

> https://deno.land/x/movie@1.0.0

Here's the video I made with it 👇🎬🎥

{% embed https://youtube.com/shorts/hpkMXIJsdaU %}