Decifer — Generate transcripts from audio using Flutter and Deepgram
Tags: hackwithdg, flutter, deepgram
Published: 2022-04-02 18:40:59

## Overview

**Decifer** is a cross-platform mobile app that helps to generate transcripts either from a voice recording or by uploading an audio file.

![Transcription Playback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uisooun6gwsfa2kup79m.gif) 

> **Try out the app:** https://play.google.com/store/apps/details?id=com.souvikbiswas.deepgram_transcribe

> Typically, for using Deepgram API you would require to maintain a server but I have made this project totally **serverless**. To know more continue reading.

Here's a brief demo of the entire app in action:

{% youtube CqRzNNgeQOE %}

## Submission Category:

Analytics Ambassadors

## Link to Code on GitHub

The entire app is open sourced - try it out and also feel free to contribute to this project 😉 :

{% github sbis04/decifer %}

## Project Description

The primary features of the app are as follows:

* Generate transcript from audio recording & audio file using [Deepgram API](https://deepgram.com/).
* Cloud-sync for syncing across multiple devices using the same account.
* Transcribe confidence map view.
* Export as PDF and share with anyone.

### Architecture

I'm using a totally **serverless** architecture for this project 🤯, let's have a look how it works:

![Decifer architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/47ixtw35g4ii66pav9z0.png)

The mobile app is created using [Flutter](https://flutter.dev/) which is integrated with [Firebase](https://firebase.google.com/). I have used Firebase **Cloud Functions** to deploy the backend code required for communicating with the Deepgram API.

> [Firebase Cloud Functions](https://firebase.google.com/docs/functions) lets you run backend code in a severless architecture.

I have deployed the following function to Firebase:

```js
const functions = require("firebase-functions");
const {Deepgram} = require("@deepgram/sdk");

exports.getTranscription = functions.https.onCall(async (data, context) => {
  try {
    const deepgram = new Deepgram(process.env.DEEPGRAM_API_KEY);
    const audioSource = {
      url: data.url,
    };

    const response = await deepgram.transcription.preRecorded(audioSource, {
      punctuate: true,
      utterances: true,
    });

    console.log(response.results.utterances.length);

    const confidenceList = [];
    for (let i =0; i < response.results.utterances.length; i++) {
      confidenceList.push(response.results.utterances[i].confidence);
    }

    const webvttTranscript = response.toWebVTT();

    const finalTranscript = {
      transcript: webvttTranscript,
      confidences: confidenceList,
    };

    const finalTranscriptJSON = JSON.stringify(finalTranscript);
    console.log(finalTranscriptJSON);

    return finalTranscriptJSON;
  } catch (error) {
    console.error(`Unable to transcribe. Error ${error}`);
    throw new functions.https.HttpsError("aborted", "Could not transcribe");
  }
});
```

The `getTranscription` function takes an audio URL, generates the transcripts using Deepgram API along with the respective confidences, and returns the data in a particular JSON format (that can be parsed within the app).

### App screens

The Flutter application consists of the following pages/screens:

* Login Page
* Register Page
* Dashboard Page
* Record Page
* Upload Page
* Transcription Page

For authenticating the user inside the app -- **Login** and **Register** pages are used. Authentication is required to generate unique accounts for users required for storing the generated transcripts to [Firestore](https://firebase.google.com/docs/firestore) and facilitate **cloud-sync**.

![Register Page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7hlznbcf4nlh8iedvt3y.png)
 
The **Dashboard Page** displays a list of all the transcripts currently present on the user's account. It also has two buttons - one for navigating to the Record Page and the other for navigating to the Upload Page.

![Dashboard Page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6xclqk0cpqpgfsd8506q.png) 

**Record Page** lets you record your audio using the device microphone and the transcribe it using Deepgram. You always have an option to re-record if you think the last recording wasn't good.

![Record Page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r52ejg10bpyozilhzmnu.png) 

From the **Upload Page**, you can choose any audio file present on your device and generate the transcript of it.

![Upload Page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gbv2c96vg4kmga247jlr.png) 

**Transcription Page** is where the entire transcript can be viewed. It has an audio-transcript synchronized playback that highlights the text transcript part with respect to the audio that is playing.

![Transcription Page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v96ekvfjm6hmx3muovud.png)
 
You can also see the **confidence map** of each of the parts of the transcript (it shows how much accurate is that part of transcript generation - _darker is higher confidence_).

![Confidence Map](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3m6ge6g885ia335neqv.gif)

You can also easily print or share the generated transcript in the **PDF** format.

![Export transcript](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3mep5k5adqafkvfqx5bb.gif)

### Deepgram

Overview of my Deepgram dashboard (completed the mission, _Get a Transcript via API or SDK_):

![Deepgram Overview](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eq5dznvzbgy1c8b4ktmz.png)

Usage analytics of the Deepgram API:

![Deepgram Usage Analytics](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/un52yb4q3qqtog1qw9vr.png)

Log of one of the API calls for transcribing from audio:

![Deepgram Logs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sutvli9mtuzne9u5ofdy.png)

<!-- ## Additional Resources -->

<!-- * Google Play Store: -->

### References

* [Deepgram API Reference](https://developers.deepgram.com/api-reference/)
* [Firebase Authentication Docs](https://firebase.flutter.dev/docs/auth/overview/)
* [Firestore Docs](https://firebase.flutter.dev/docs/firestore/overview)
* [Cloud Functions](https://firebase.flutter.dev/docs/functions/overview)