Add live captions to a video call app with daily-js
Tags: javascript, webdev, transcription, video
Published: 2022-04-25 18:34:40

---
title: Add live captions to a video call app with daily-js
description: Use the daily-js transcription API to get subtitles with every video call
published: true
tags: javascript, webdev, transcription, video
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0zcxc0k02xv49uvw1lqw.png
canonical_url: https://www.daily.co/blog/add-live-captions-to-your-daily-prebuilt-calls/
---

## Introduction
At Daily, we love seeing what developers build in whatever way works best for them. We offer [Daily Prebuilt](https://docs.daily.co/guides/products/prebuilt) – our embeddable video call UI – to get started with just a couple lines of code or you can build a totally custom experiences with Daily’s [call object](https://docs.daily.co/guides/products/call-object). Our goal is to support everyone’s use case and skill level by making browser-based video calls easy to integrate and customize.

As part of making the call experience better for everyone, we have introduced the ability to add live captions to Daily domains with our [`startTranscription()`](https://docs.daily.co/reference/daily-js/instance-methods/start-transcription) instance method in partnership with [Deepgram](https://deepgram.com/). 

This highly requested feature has many uses. They include being able to:
* Introduce wider accessibility options within calls
* Provide an “instant replay” (e.g. What did they just say?)
* Produce a great way to generate meeting notes

![video call participant's audio being transcribed in the demo app](https://www.daily.co/blog/content/images/2022/04/transcription.gif)

This tutorial focuses on pairing transcription with Daily Prebuilt. We already have a thorough tutorial on [adding transcription to a custom Daily call](https://www.daily.co/blog/add-live-transcription-to-a-daily-call-with-our-newest-api/), which you should definitely check out! Today’s tutorial includes a demo that walks through how to add transcription alongside a Daily Prebuilt call.

If you are like spoilers and want to see what we’re building today, you can jump straight into the [prebuilt-transcription](https://github.com/daily-demos/prebuilt-transcription) code and also try a [live demo](https://prebuilt-transcription.vercel.app/) 

**Note:** You will have to connect your Daily and Deepgram accounts, as outlined in the demo’s README and in this tutorial, to fully experience the live demo.

## What’s the plan? 
In this tutorial, we will cover:
* Getting set up with Daily and Deepgram
* Embedding Daily Prebuilt in a Next.js app
* Creating a transcription component
* Adding buttons to start and stop transcription methods
* Loading the live captions
* Optimizing the app for large amounts of text
* Downloading the transcript
## Prerequisites

Because transcription is a shared service between Daily and Deepgram, there is a bit of set-up involved to get both services running with each other. But the good news is that you can set it and forget it, because it’s just a one-time step!

_Note: Daily does not charge for transcription services and Deepgram offers a free $150 credit upon [sign-up](https://console.deepgram.com/signup), so there is no cost associated with this tutorial._

To walk through this tutorial, you will first need to [create a Daily account](https://dashboard.daily.co/signup) and a [Deepgram account](https://console.deepgram.com/signup). Once you have an account and are logged into the [Daily Dashboard](https://dashboard.daily.co), you can [create a new Daily room](https://dashboard.daily.co/rooms/create) or use our [REST API](https://docs.daily.co/reference/rest-api/rooms).

To set up transcription, you will have to [`enable_transcription` on your Daily domain](https://docs.daily.co/reference/rest-api/your-domain/config#enable_transcription). 

Essentially, you'll need your Daily API key, available in the [Daily dashboard](https://dashboard.daily.co/developers), and your [Deepgram API key](https://console.deepgram.com/) to update your domain settings, like so:

```bash
curl --request POST \
    --url https://api.daily.co/v1/ \
    --header 'Accept: application/json' \
    --header 'Authorization: Bearer YOUR_DAILY_API_KEY' \
    --header 'Content-Type: application/json' \
    --data '{"properties": { "enable_transcription": "deepgram:YOUR_DEEPGRAM_API_KEY" }}'
```

## Setting up the demo
Head on over to the [prebuilt-transcription](https://github.com/daily-demos/prebuilt-transcription) GitHub repository and fork the repo to follow along with the rest of this post.

After forking and navigating to the `prebuilt-transcription` folder, install the dependencies:

```bash
npm i
```

And run the dev server:

```bash
npm run dev
```

Locally, open [http://localhost:3000](http://localhost:3000) in your browser.

This demo is based on the [Next.js](https://nextjs.org/) React framework, starting with the [`create-next-app`](https://nextjs.org/learn/basics/create-nextjs-app/setup) template builder. This tutorial also uses TypeScript. If you are new to TypeScript, no worries! Because it’s built on top of JavaScript, it looks very similar with a few additional features and syntax.

## Staging the `[room]`

The most interesting part of our codebase lives in `pages/[domain]/[room].tsx`, so let’s start there. And yes, those brackets are part of the file names – this allows us to create URLs dynamically in Next.js.

When we load the page, we want to build and start the call right away with the parameters we retrieve from the page URL. To do this, we create a `useCallback` function:

```javascript
/*
   Set up the Daily call and event listeners on page load
 */
 
 const startCall = useCallback(() => {
   const iframe = document.getElementById("callFrame");
   const newCallFrame = DailyIframe.wrap(iframe as HTMLIFrameElement, {
     showLeaveButton: true,
   });
   setCallFrame(newCallFrame);
 
   newCallFrame.join({
     url: url,
   });
 
   newCallFrame.on("joined-meeting", (ev) => {
     let ownerCheck = ev?.participants.local.owner as boolean;
     setIsOwner(ownerCheck);
   });
 
  // snip snip - some more event listeners here 
  // we’ll come back to this section later!
 
 }, [url]);
```

The above `startCall` function loads when the page loads, via a React `useEffect`:

```javascript
 useEffect(() => {
   startCall();
 }, [startCall]);
```

This creates and joins a call built from the URL parameters.
## Incorporating the Daily Prebuilt iframe
Now that the app framework is up and running, let’s add Daily Prebuilt. Since Daily Prebuilt is an embeddable video call UI, Daily has already done most of the video-related work for you. That means this part will be short.

From the aforementioned `[room].tsx` page, we load the `<CallFrame>` component. The full component looks like this:

```javascript
import styles from "../styles/CallFrame.module.css";
 
const CallFrame = () => (
 <div className={styles.callFrameContainer}>
   <iframe
     id="callFrame"
     className={styles.callFrame}
     allow="microphone; camera; autoplay; display-capture"
   ></iframe>
 </div>
);
 
export default CallFrame;
```

The few lines of styling imported from `../styles/CallFrame.module.css` allow the Daily Prebuilt iframe to take up most of the screen:

```css
.callFrameContainer {
 width: 100%;
 height: 80%;
}
 
.callFrame {
 width: 100%;
 height: 100%;
 min-height: 80vh;
 border: 0;
}
```

Without this styling for the `<iframe>` and its container, Daily Prebuilt defaults to taking up only a small amount of space on the page. You can change the styling however you like and Daily Prebuilt will fit within those constraints.

_Note: We are using [`wrap()`](https://docs.daily.co/reference/daily-js/factory-methods/wrap#main) to add Daily Prebuilt to an existing `<iframe>`, but you could also use the [`createFrame`](https://docs.daily.co/reference/daily-js/factory-methods/create-frame) method to make a new `<iframe>`, style that frame, and add it to the page._

![Video call UI when a token or call owner is not present and transcription can't be used](https://www.daily.co/blog/content/images/2022/04/transcription2.png)

## Transcription component
Now that we have Daily Prebuilt loaded on the page, let’s start implementing transcription by adding a component to store our buttons and transcript. 

From our `[room].tsx`,  we reference the `Transcription` component and pass some managed state to the component: 
* `callFrame`: passes the Daily call frame object, which allows the component to start and stop transcription
* `newMsg`: sends each new transcripted message to the component for showing the text in the transcript window
* `owner`: this boolean tells the component whether the current user is or isn’t a [room owner](https://docs.daily.co/reference/rest-api/meeting-tokens#participant-details-meeting-permissions-and-available-features)
* `isTranscribing`: this boolean tells the component that Daily is or isn’t currently transcribing.

```javascript
 <Transcription
   callFrame={callFrame}
   newMsg={newMsg}
   owner={isOwner}
   isTranscribing={isTranscribing}
 />
```

In our `Transcription` component (defined in `components/Transcription.tsx`), we have a button that toggles the option to start or stop transcription based on whether transcription is currently active according to Daily (we’ll come back to that in a second):

```javascript
<button
 disabled={!owner}
 onClick={() => {
   isTranscribing ? stopTranscription() : startTranscription();
 }}
>
 {isTranscribing ? "Stop transcribing" : "Start transcribing"}
</button>
```

If the meeting participant is not an owner, this button will be disabled along with a message explaining that only meeting room owners can start transcription.

This button utilizes these two simple functions:

```javascript
 async function startTranscription() {
   callFrame?.startTranscription();
 }
 
 async function stopTranscription() {
   callFrame?.stopTranscription();
 }
```

How do these functions know if transcription is happening or not? For that, we jump back to `[room].tsx`. Earlier in the post, we looked at the basic structure of the `startCall` function. In our demo, this function also has a few lines dedicated to Daily event listeners. We are listening to a few Daily-emitted events that help us shape the video call experience. Two of these events are [`transcription-started`](https://docs.daily.co/reference/daily-js/events/transcription-events#transcription-started) and [`transcription-stopped`](https://docs.daily.co/reference/daily-js/events/transcription-events#transcription-stopped) events.

When those events are emitted, we know to update the React state to set `isTranscribing` to its correct boolean value.

```javascript
   newCallFrame.on("transcription-started", () => {
     setIsTranscribing(true);
   });
 
   newCallFrame.on("transcription-stopped", () => {
     setIsTranscribing(false);
   });
```

_Note: You can also use our new [Daily React Hooks](https://www.daily.co/blog/introducing-the-new-daily-react-hooks-library/) library to more quickly connect your React-based app with Daily’s JavaScript API!_

## Adding transcription
Now that we are able to start and stop transcription, we need to add the transcripts to the page. Our transcripts come in from Daily via an [`”app-message”`]() event. For that, we need another event listener within our `startCall` function. This checks whether each `”app-message”` came from the ID of “transcription” and whether it is a full sentence (that’s what `data.is_final` is doing below). When we have a message, we save the message as an object with the author’s username, the text transcription, and a timestamp.

```javascript
   newCallFrame.on(
     "app-message",
     (msg: DailyEventObjectAppMessage | undefined) => {
       const data = msg?.data;
       if (msg?.fromId === "transcription" && data?.is_final) {
         const local = newCallFrame.participants().local;
         const name: string =
           local.session_id === data.session_id
             ? local.user_name
             : newCallFrame.participants()[data.session_id].user_name;
         const text: string = data.text;
         const timestamp: string = data.timestamp;
 
         if (name.length && text.length && timestamp.length) {
           setNewMsg({ name, text, timestamp });
         }
       }
     }
   );
```

We need some React state to hold messages, so we set up a `const` where we instantiate this state as an empty array to hold incoming message objects.

```javascript
const [messages, setMessage] = useState<Array<transcriptMsg>>([]);
```

This is essentially all that needs to be done to get transcription on the page. You can loop through this array of messages and add them to the screen, or you can add each new message to the screen as it arrives. However, there’s one extra step worth taking to optimize your app for all of these messages, and we’ll see how that works in the next section.

_Note: Transcript messages are ephemeral. They are only available for the message the user has received while they are in the room. If you refresh your page, you’ll lose the transcripts. Similarly, new users will only see a transcript for conversations that have taken place since they’ve joined and not a history._

![Transcribed text scrolling up and down](https://www.daily.co/blog/content/images/2022/04/transcription3.gif)

## Optimize your window
Seeing transcripts appear on the screen is super fun, but it can quickly slow down browser windows with the addition of so many DOM elements getting added to the screen. Below, we’ll cover not just how we add transcript messages to our page, but also how to do it in a way that is efficient and not overwhelming to anyone’s browser.

To help with this, we need to add two dependencies to our app: [`react-window`](https://www.npmjs.com/package/react-window) and [`react-virtualized-auto-sizer`](https://www.npmjs.com/package/react-virtualized-auto-sizer). These libraries help us by loading only the most recent messages. Instead of loading the entire array of message objects as HTML, the DOM only loads the small part of the data set visible in the window. This virtualization technique prevents poor performance caused by an overloaded browser tab holding too much data in memory. Users can still scroll up and see previous messages which are loaded as needed when requested.

We have established `const`s for the transcript list and rows that instantiate as empty objects.

```javascript
 const listRef = useRef<any>({});
 const rowRef = useRef<any>({});
```

We add new messages received from the parent `[room]` page to an array. We also have a small function that keeps the array of messages moving to the bottom (most recent) element every time a message is received.

```javascript
 useEffect(() => {
   setMessage((messages: Array<transcriptMsg>) => {
     return [...messages, newMsg];
   });
 }, [newMsg]);
 
 useEffect(() => {
   if (messages && messages.length > 0) {
     return () => {
       scrollToBottom();
     };
   }
 }, [messages]);
```


For each row, we call a formatting function. It structures the transcript in the style of “Message Author: Message Text” on the left and Timestamp trimmed to a local time only on the right (styled with CSS, the handy-dandy `float:right`).

```javascript
 function Row({ index, style }: rowProps) {
   const rowRef = useRef<any>({});
 
   useEffect(() => {
     if (rowRef.current) {
       setRowHeight(index, rowRef.current.clientHeight);
     }
   }, [rowRef]);
 
   return (
     <div style={style}>
       {messages[index].name && (
         <div ref={rowRef}>
           {messages[index].name}: {messages[index].text}
           <span className={styles.timestamp}>
             {new Date(messages[index].timestamp).toLocaleTimeString()}
           </span>
         </div>
       )}
     </div>
   );
 }
```

Our rendered transcript block then looks like this, with each loaded row wrapped in the `react-window` List and `react-virtualized-auto-sizer` AutoSizer elements.

```javascript
<AutoSizer>
 {({ height, width }) => (
   <List
     height={height}
     width={width}
     itemCount={messages.length}
     itemSize={getRowHeight}
     ref={listRef}
   >
     {Row}
   </List>
 )}
</AutoSizer>
```

## Download

The transcripts collected in this app are not available after the call concludes, so downloading them is helpful if you want to use them later.

To do that, we 	need to prepare a chat file with all of the text, not just the text currently virtualized on the screen.

We have already seen that we are using React state to collect and set messages. For preparing a plain text file with the transcript inside, we will add a `transcriptFile` state that instantiates as an empty string.

```javascript
const [transcriptFile, setTranscriptFile] = useState<string>("");
```


Next, let’s set up a `useEffect` to style the transcript in a way that works best for reviewing later. Unlike the live transcript where we have the timestamp on the right and set to local time only, this includes the full timestamp and date for every message.

```javascript
 /*
   Allow user to download most recent full transcript text
 */
 
 useEffect(() => {
   setTranscriptFile(
     window.URL.createObjectURL(
       new Blob(
         messages.map((msg) =>
           msg.name
             ? `${msg.timestamp} ${msg.name}: ${msg.text}\n`
             : `Transcript\n`
         ),
         { type: "octet/stream" }
       )
     )
   );
 }, [messages]);
```

This link will get the most recent full transcript and by default save it as a file called `transcript.txt`, although this can be changed later by the user.

```javascript
<a href={transcriptFile} download="transcript.txt">
  Download Transcript
</a>
```

## Conclusion
And there you have it! Using Daily Prebuilt and our new Transcription API with Deepgram, it’s not too much work to add a live transcript to your meetings. From what we’ve shown in this demo, you can easily add different styles (including to the Daily Prebuilt window itself by [customizing with your own color themes](https://docs.daily.co/guides/daily-prebuilt-features/customizing-daily-prebuilt-calls-with-color-themes#main))

We would love to see what you’ve built using Daily. Reach out to us anytime at [help@daily.co](mailto:help@daily.co)!