Video Transcription using Cloudinary Add-on in a Nuxt 3 Application
Tags: webdev, javascript, tutorial, showdev
Published: 2023-02-06 03:42:52

Video transcription is the process of converting the spoken words in a video into written text. This can be done automatically using speech recognition technology or manually by transcribers. The transcribed text is usually synchronised with the video, allowing viewers to read along as they watch.
There are a variety of use cases for video transcription, including:

- Accessibility: Transcription can make videos more accessible to people with hearing impairments or those who speak different languages.
- Searchability: Transcribed videos can be searched and indexed, making it easier for viewers to find specific information within the video.
- Subtitling: Transcription can be used to create subtitles for videos, making them more accessible to a broader audience.
- Language Learning: Transcription can be used as a tool for language learning, allowing students to read along with native speakers and improve their listening and comprehension skills.

This implies that video transcription is a necessity for improved accessibility. As a result, several SaaS companies, such as Cloudinary, offer a method to transcribe videos smoothly.

[Cloudinary](https://cloudinary.com/?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost), a cloud-based platform for managing and optimising media assets such as images and videos, also offers a video transcription add-on. The add-on converts the audio in our videos to text automatically. It applies an effective AI algorithm to our videos using Google’s Cloud Speech API to generate the best possible speech recognition results. These generated texts can be overlayed on our video as subtitles using Cloudinary video transformations.

In this article, we will build a Nuxt application that uploads a video to Cloudinary and applies the Cloudinary transcription add-on on upload. Then we will use the Cloudinary transformation API to overlay the AI-generated text on the video. Finally, we will make the transformed video downloadable.


## Application Repository and Demo

**Repo**: https://github.com/yhuakim/nuxi-transcription
**Demo**: https://nuxi-transcription.netlify.app/


## Prerequisites
- A Cloudinary free tier [account](https://cloudinary.com/users/register_free?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost#gsc.tab=0) and valid API keys.
- Good knowledge of Nuxt version 3.


## Registering the Google AI Video Transcription Addon

After creating a free tier account, enable the Google AI video transcription addon to use the Cloudinary transcription endpoint. We can do this by following the steps below:

1. Navigate to the addon tab, then click the Google AI video transcription card as shown in the image below:
![Cloudinary Addon setup; step1](https://paper-attachments.dropboxusercontent.com/s_2C43D2B4B2CE1C8CA47117A06FCB6B9858E89FCF5E799A3F5B847B0F57D9D80A_1674745699033_image.png)

2. The preceding step will take us to the screen below, where we can select the Free plan link. This will enable our free plan.
![Cloudinary Addon setup; step2](https://paper-attachments.dropboxusercontent.com/s_2C43D2B4B2CE1C8CA47117A06FCB6B9858E89FCF5E799A3F5B847B0F57D9D80A_1674746267093_image.png)


Now, we have successfully registered the Google AI video transcription addon.


## Setting Up a Development Environment

To set up a development environment, we will need to run the following commands in the terminal:

    npx nuxi init video-transcription && cd video-transcription

This command creates a Nuxt project in a directory called `video-transcription` and navigates into the directory.

Next, we need to install the Nuxt3 dependencies by running the following command in the terminal:

    yarn install

**Installing Dependencies**
After initializing a Nuxt project, we will add other dependencies necessary for the current project. These dependencies include:

**Cloudinary:** A Node.js SDK for media management. We can add this package by entering the following command:

    yarn add cloudinary

**[File saver](https://github.com/eligrey/FileSaver.js?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost#readme):** A package to download the transcribed video to the storage device. This can be added using the following command:

    yarn add file-saver

**[TailwindCSS](https://tailwindcss.com/docs/installation?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost):**  A CSS framework to assist with the page styles. To use this, we can run the following command:

    yarn add -D tailwindcss postcss autoprefixer
    npx tailwindcss init -p

Then, we can follow the instructions [here](https://tailwindcss.com/docs/guides/nuxtjs?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost#3) for a complete guide to set up TailwindCSS in a Nuxt3 application.

## Building Out The User Interface

To create the UI of this project,  we will split it into two major sections namely:

- The Form handling section: This will handle the user’s input, the video file and the upload to our Cloudinary endpoint.
- The Transcribed video preview: Here, the returned transcribed video is displayed to the user with a download button.

Before we start writing the code, let's adjust our file structure as follows:

1. Navigate to the project directory and create a `pages` folder.
2. Then, open the `app.vue` file and add `<NuxtPage />` component as shown below, and then run `yarn dev -o` to start the development server.

```html
<template>
    <div>
       <NuxtPage />
    </div>
</template>
```

**Form Handling**
Now let’s create a file in the `pages` folder called `index.vue` then, add the following snippet

```vue
<!-- pages/index.vue -->
    <template>
      <div class="max-w-[60rem] grid justify-center items-center mx-auto py-4">
        <header class="text-4xl font-bold text-gray-700 text-center">
          <h1>Video Transcription App</h1>
        </header>
        <main class="py-5">
          <section class="py-5 flex">
            <form @submit="handleSubmit">
              <div class="upload">
                <label
                  for="video-file"
                  class="block mb-2 text-sm font-medium text-gray-500"
                  >Upload a Video file</label
                >
                <input
                  type="file"
                  @change="handleChange"
                  name="video-file"
                  id="video-file"
                  class="g-gray-50 border border-gray-300 text-gray-900 text-sm 
                  rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
                />
              </div>
              <button
                v-if="selected !== null"
                type="submit"
                class="text-white bg-gradient-to-r from-blue-500 via-blue-600 
                to-blue-700 hover:bg-gradient-to-br focus:ring-4 focus:outline-none 
                focus:ring-blue-300 dark:focus:ring-blue-800 font-medium rounded-lg 
                text-sm px-5 py-2.5 text-center mr-2 mb-5 mt-2"
              >
                Upload
              </button>
            </form>
          </section>
        </main>
      </div>
    </template>

```

The snippet above is a markup for our Form, which consists of an `input` element with a type of `file` and a `button` with a type of `submit`. The `submit` button is configured to be visible only when a user has selected a valid input.

Next, let’s create the `handleSubmit` and `handleChange` methods that we have defined in the markup above. Let’s add the following snippet just above our `template` markup

```vue
<!-- pages/index.vue -->
    <script setup>
    const selected = useState("selected", () => null);
    const videoUrl = useState("videoUrl", () => "");
    const downloaded = useState("downloaded", () => false);
    
    const handleChange = (e) => {
      if (e.target.files && e.target.files[0]) {
        const i = e.target.files[0];
        let reader = new FileReader();
        reader.onload = () => {
          let base64String = reader.result;
          selected.value = base64String;
        };
        reader.readAsDataURL(i);
      }
    };
    
    const handleSubmit = async (e) => {
      e.preventDefault();
      try {
        const body = JSON.stringify(selected.value);
    
        const config = {
          headers: {
            "Content-Type": "application/json",
          },
        };
        const { data } = await useFetch(`/transcribe`, {
          method: "POST",
          headers: config.headers,
          body,
        });
        videoUrl.value = JSON.parse(data.value.data);
      } catch (error) {
        console.error(error);
      }
    };
    </script>

```

The snippet above shows how we manage our states, monitor and process the user’s input using the `handleChange` method and what to do with the `file` once the user submits it. To manage our states we will use the Nuxt provided `useState` composable. To learn more about the `useState` composable [visit here](https://nuxt.com/docs/getting-started/state-management?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost).

In the `handleChange` method, we will convert the user-selected video file into a data url format using Javascript’s built-in `FileReader()` constructor. Then we will assign it to the `selected` state.

Then, in the `handleSubmit` method, we will:

 - prevent the default action of the onSubmit event by calling the `preventDefault()` function
 - send a `POST` request to our server route (`/transcribe`), with the value of the `selected` state as the `body` of the request
 - now assign the response to our `videoUrl` state

**Creating the Server Route**
From the above snippet, we are making a `POST` request to the `/transcribe` route but we have not created this route. To create this route, let’s navigate to the project directory and create a folder named `server`. Inside this folder, let’s create another folder called `routes` and create a file called `transcribe.js`. 

In the `transcribe.js` file, we will handle two major logic:

- Upload and Transcription: Here we will use the Cloudinary upload API to upload the user-selected video file. Also, we will ask Cloudinary to transcribe this video using the transcription addon.
- Video Transformation: Here we will use the Cloudinary video transformation API to overlay the generated text on the video.

First, let’s add the following snippet to `/transcribe.js`:

```javascript
    import { v2 as Cloudinary } from 'cloudinary';
    import { parse } from 'path'
    
    export default defineEventHandler(async (event) => {
        const { name, path } = await readBody(event)
        const filename = parse(name).name
    
        try {
            Cloudinary.config({
                cloud_name: process.env.CLOUD_NAME,
                api_key: process.env.API_KEY,
                api_secret: process.env.API_SECRET,
                secure: true
            });
    
            await Cloudinary.uploader.upload(path,
                {
                    resource_type: "video",
                    public_id: `demo/${filename}`,
                    raw_convert: "google_speech:srt:vtt"
                },
                function (error, result) {
                    if (result) {
                        return result
                    }
                });
    
        } catch (error) {
            console.log(error);
        }
    })

```

In the snippet above, we

- imported the `cloudinary` and `path` packages
- created a handler function for our server logic which takes an event parameter.
- destructured the `name` and `path` from the body of our request using Nitro’s built-in `readBody` query and passing in the `event` parameter. 
- trimmed off the file extension from the user-selected file name using the `parse` method we imported, by passing in the file name and then retrieving only the name attribute.
- set up a Cloudinary instance by passing in our `cloud_name`, `api_key` and `api_secret`, inside a try-catch statement.
- used the `Upload` method provided by the Cloudinary instance to upload the video file located at the path stored in our `path` variable. Additionally, we will set the `raw_convert` option to `google_speech:srt:vtt`. This will trigger an automatic transcription of the video as it uploads.

Once this process is completed, we have one more task to perform, and that’s to overlay the generated text file on our video. To do this, we will add the following snippet just below our upload method.

```javascript
    // server/routes/transcribe.js
    const transcribedVideo = Cloudinary.url(`demo/${filename}`, {
          resource_type: "video",
          loop: false,
          controls: true,
          autoplay: true,
          fallback_content: "Your browser does not support HTML5 video tags",
          transformation: [
              {
                  overlay: {
                      resource_type: "subtitles",
                      public_id: `demo/${filename}.en-US.srt`
                  }
              },
              { flags: "layer_apply" }
          ]
      })
    return {
        statusCode: 200,
        data: JSON.stringify(transcribedVideo)
    };

```

In the above snippet, we;

- used the Cloudinary `url` endpoint to retrieve the uploaded video by its name, and then we applied an `overlay` transformation to it. The `overlay` will be of `resource_type` **subtitles**, then we will provide the location of our generated text using the `public_id` option.
- returned the `url` generated from our Cloudinary transformation as a `JSON` string to the client side with a `statusCode` of `200`.

Now we have successfully created our server logic and can post and retrieve data from our client side, let’s display our transcribed video to the user.

**Displaying The Transcribed Video and Adding a Download Button**
Here we will strive to display the transcribed video as returned from our server logic to our users. Additionally, we will provide a download button, so the user can save the video to their local storage.

Now let’s navigate to our `pages/index.vue` file and add the following snippet just below the Form section.

```vue
    <section class="transcribed-preview">
      <!-- preview Transcribed video here -->
      <div v-if="videoUrl === ''" class="">
        Your transcribed video will appear here
      </div>
      <div v-else class="max-w-[35rem] shadow-2xl">
        <video autoplay controls className="mb-5 w-100">
          <source
            :src="videoUrl ? `${videoUrl}.webm` : ''"
            type="video/webm"
          />
          <source :src="videoUrl ? `${videoUrl}.mp4` : ''" type="video/mp4" />
          <source :src="videoUrl ? `${videoUrl}.ogv` : ''" type="video/ogg" />
        </video>
      </div>
      <button
        v-if="videoUrl !== ''"
        @click="handleDownload"
        class="text-white bg-gradient-to-r from-blue-500 via-blue-600 to-blue-700 
        hover:bg-gradient-to-br focus:ring-4 focus:outline-none focus:ring-blue-300 
        dark:focus:ring-blue-800 font-medium rounded-lg text-sm px-5 py-2.5 
        text-center mr-2 mb-2"
        :disabled="downloaded ? true : false"
      >
        Download Video
      </button>
    </section>

```

The above snippet only displays when we have a response from our server. Now let’s handle our download button logic. To do this we will add the following snippet inside our `script` element.

```javascript
    import { saveAs } from "file-saver";
    
    const handleDownload = () => {
      if (videoUrl.value !== "") {
        saveAs(videoUrl.value, "transcribed video");
        downloaded.value = true;
      }
    };

```

The method above uses the `saveAs` method we imported to initiate a download when the button is clicked. The `saveAs` method takes in two parameters; the first is the `url` of the file and the second is the name of the file that will be saved. 

Finally, we will have the following screen once we have successfully executed all these procedures.

{% embed https://www.dropbox.com/s/yupbb2wrfkhb0xy/video-transcription-demo.mp4?dl=0 %}

## Conclusion

This article shows a managed method to implement video transcription with Cloudinary in a Nuxt version 3 application. It exposes the usage of server routes to protect and manage our API keys. Additionally, we explored the use of Nuxt3 reactive state management to manage our application states.


## Resources

The following are resources for further reading and guides to set up our next project.

- [Cloudinary Documentation](https://cloudinary.com/documentation?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost)
- [Cloudinary NodeSDK](https://cloudinary.com/documentation?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost)
- [Getting started with Nuxt3](https://nuxt.com/docs/getting-started?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost)
- [TailwindCSS configuration guide for Nuxt 3](https://tailwindcss.com/docs/guides/nuxtjs?utm_source=yhuakim&utm_medium=social-media&utm_campaign=blogpost#3)