Journal your ideas and experiences with your voice: Voice Journal
Tags: cloudflarechallenge, devchallenge, ai, nextjs
Published: 2024-04-13 18:08:02

*This is a submission for the [Cloudflare AI Challenge](https://dev.to/devteam/join-us-for-the-cloudflare-ai-challenge-3000-in-prizes-5f99).*

## What I Built

Voice Journal is a journal app which transcribes your natural speech to what I call a _voice note_ 🔉. Apart from this, it also stores your voice recordings for playback ⏯️. 

It has built-in AI chat integration, so you can ask anything about your notes, like _list the activities I did today_. You can also generate a concise summary of your voice note.

## Demo

<!-- Share a link to your deployed solution on Workers or Pages, and add a screenshot or video embed of your app here. -->

Here, [try the live version of Voice Journal](https://voicejournal.pages.dev/) deployed on Cloudflare Pages and AI Workers.

![VoiceJournal Demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fv5dfvuig99e6lfx2plk.gif)


## My Code

<!-- Show us the code! Share a public link to your repo and be sure to include a README file with installation instructions. We also encourage you to add a license for your code.  --> 

Here’s the [GitHub repository](https://github.com/prashikm/voice-journal)

The project is structured into two directories: web for frontend and worker for AI workers.

## Journey

<!-- Tell us about your process, the task types and models you used, what you learned, anything you are particularly proud of, what you hope to do next, etc. -->

It was my first time using Cloudflare AI Workers, and I had an amazing developer experience working with it. I started building the app with AI Workers and used Hono as a routing library.

Apart from this, I was getting an error while working with the `whisper` model via the REST API; thus, later, I tried the same but with Cloudflare AI Bindings for Workers and it worked.

Thus, I had to deploy the workers separately for the same issue.

Build the frontend with Next.js, Tailwind CSS and shadcn components, integrated the AI Worker, and finally deploy it on Cloudflare Pages.

I have integrated three AI models, here’s the detail for each:

The **whisper** model is used for generating transcripts from audio; it takes an array buffer converted to an 8-bit unsigned integer for audio processing.

For text-generation, integrated **mistral-7b-instruct-v0.1** model.

To provide a summary of voice notes, the **bart-large-cnn** model is used, which tries to summarize the recorded voice notes. _I think it needs more fine-tuning for better results_.

The most challenging part of this whole building process was handling the audio data format and how to process it, as it was my first time dealing with it.

The **future plan** is to connect Cloudflare R2 object storage to store the audio files and fine-tune the summarization model.

**Multiple Models and/or Triple Task Types**

<!-- Let us know if your project utilized multiple models per task and/or if your project used three or more task types. If so, you may qualify for our additional prize categories! If not, please remove this section. -->

The Voice Journal utilizes three models for performing the tasks:

1. **whisper** for speech to text conversion
2. **mistral-7b-instruct-v0.1** for text generation
3. **bart-large-cnn** to summarize the notes
 
<!-- Team Submissions: Please pick one member to publish the submission and credit teammates by listing their DEV usernames directly in the body of the post. -->

<!-- Don't forget to add a cover image or an additional tag to this post (if you want). -->

<!-- Thanks for participating! -->