Create AI Embedding app using Supabase & SvelteKit
Published: 2024-04-24 02:30:26

---
title: Create AI Embedding app using Supabase & SvelteKit
published: true
date: 2023-10-10 05:53:16 UTC
tags: 
canonical_url: https://sucipto.id/2023/10/10/create-ai-embedding-app-using-supabase-sveltekit/
---

A few weeks back, Supabase shared an exciting customer story about the [Quivr app](https://supabase.com/customers/quivr). Quivr allows users to have dynamic conversations with their second brain or knowledge base, effortlessly importing content from various formats like text, PDFs, web page URLs, and audio. Quivr’s backend runs on Python, utilizing FastAPI and Langchain, while the frontend is powered by Next.js. In this post, we’ll simplify the process by creating a Quivr-like app using SvelteKit and Supabase.

We’ll kick off with my previous Supabase project, Supalytic (a runner-up in the Supabase hackathon, [check it out here](https://github.com/suciptoid/supalytic)). It’s already configured with Supabase auth and a database tailored for SvelteKit SSR.

For a foundational understanding of `pg_vector` and embedding integration with Supabase, refer to the informative [blog post](https://supabase.com/blog/openai-embeddings-postgres-vector) by Supabase. It’s an excellent resource to get started.

## F﻿eature Plan

In building this app, our aim is a simple knowledge base. Users can input and store text, generate embeddings using OpenAI, save the vectors on Supabase, and use them later for contextual chat. To keep things straightforward, we’re focusing on text content for now.

## Building App

As mentioned earlier, I’m utilizing the starter project from Supalytic. This saves me from setting up the web app portion since it already comes equipped with Tailwind CSS, Supabase authentication, and a pre-configured database. I’ve streamlined the code by removing any unrelated components from Supalytic.

The next step involves creating a migration using the Supabase CLI. You can find detailed instructions in the Supabase documentation [here](https://supabase.com/docs/reference/cli/supabase-migration-new). Below is our finalized migration script for the app:

```
-- Enable the pgvector extension to work with embedding vectors
CREATE EXTENSION IF NOT EXISTS vector;

-- Create a table to store your documents
CREATE TABLE documents (
 id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
 knowledge_id UUID,
 content TEXT, 
 embedding VECTOR(1536) -- 1536 works for OpenAI embeddings, change if needed
);

-- Add a HNSW index for the inner product distance function
CREATE INDEX ON documents
USING hnsw (embedding vector_ip_ops);

-- Create a table to store knowledge entries
CREATE TABLE knowledges (
 id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
 user_id UUID,
 title VARCHAR,
 created_at TIMESTAMP DEFAULT NOW()
);

-- Create or replace a function to match documents with knowledge references
CREATE OR REPLACE FUNCTION match_documents (
 knowledge_id UUID,
 query_embedding VECTOR(1536),
 match_threshold FLOAT,
 match_count INT
)
RETURNS TABLE (
 id UUID,
 knowledge_id UUID,
 content TEXT,
 similarity FLOAT
)
LANGUAGE SQL STABLE
AS $$
 SELECT
 documents.id,
 documents.knowledge_id,
 documents.content,
 1 - (documents.embedding <=> query_embedding) AS similarity
 FROM documents
 WHERE documents.knowledge_id = knowledge_id
 AND 1 - (documents.embedding <=> query_embedding) > match_threshold
 ORDER BY similarity DESC
 LIMIT match_count;
$$;

```

With the recent release of **HNSW** index support by Supabase (check it out [here](https://supabase.com/blog/increase-performance-pgvector-hnsw)), we’re excited to implement it in our app for enhanced performance. Let’s leverage this feature to optimize the efficiency of our application.

The next crucial step is to generate embeddings when content is added to the knowledge base. Here’s a snippet of the action handler from our project:

```
import type { Actions } from './$types';
import OpenAI from 'openai';
import { env } from '$env/dynamic/private';

export const actions = {
 default: async ({ request, params, locals: { db } }) => {
 const form = await request.formData();
 const content = form.get('content')?.toString() ?? '';

 const openai = new OpenAI({ apiKey: env.OPENAI_KEY });

 const embeds = await openai.embeddings.create({
 input: content,
 model: 'text-embedding-ada-002'
 });

 const [{ embedding }] = embeds.data;

 const response = await db.from('documents').insert({
 content,
 embedding,
 knowledge_id: params.id
 });
 }
} satisfies Actions;

```

At this stage, we’ve successfully stored knowledge base content with embedding vectors in the Supabase database. Moving forward, our next step is to incorporate queries from the knowledge base into the user’s chat context. Let’s seamlessly integrate these elements for a more dynamic and interactive experience.

To enhance interactivity, we’ll utilize the OpenAI stream response. Luckily, SvelteKit simplifies the handling of such streams through Server-Sent Events (SSE), facilitating the seamless transmission of streamed responses from the server to the client UI. Here’s how it looks in TypeScript:

```
import OpenAI from 'openai';
import type { RequestHandler } from './$types';
import { encode } from 'gpt-tokenizer/esm/model/gpt-3.5-turbo';
import { env } from '$env/dynamic/private';

export const POST: RequestHandler = async ({ request, params, locals: { db } }) => {
 const form = await request.json();
 const content = form.prompt;

 const openai = new OpenAI({ apiKey: env.OPENAI_KEY });

 const embeds = await openai.embeddings.create({
 input: content,
 model: 'text-embedding-ada-002'
 });

 const [{ embedding }] = embeds.data;

 const { data: documents, error } = await db.rpc('match_documents', {
 knowledge_id: params.id,
 query_embedding: embedding,
 match_threshold: 0.78, // Choose an appropriate threshold for your data
 match_count: 10 // Choose the number of matches
 });
 console.log('error', error);

 const maxToken = 1500;
 let tokenCount = 0;
 let contextText = '';

 console.log('documents count', documents?.length);

 if (documents) {
 for (let i = 0; i < documents.length; i++) {
 const doc = documents[i];
 const encoded = encode(doc.content);
 console.log('doc id', doc.id, 'token length:', encoded.length);
 tokenCount += encoded.length;

 if (tokenCount > maxToken) {
 break;
 }

 contextText += `${doc.content.trim()}\n---\n`;
 }
 }

 console.log('context', contextText);
 console.log('----\n\n');

 let prompt = `You are a very enthusiastic Assistant who loves to help people! Given the following sections from the personal knowledge base, answer the question using only that information, outputted in markdown format. If you are unsure and the answer is not explicitly written in the documentation, say "Sorry, I don't know how to help with that".`;
 prompt += `

 Context sections: 
 ${contextText}

 Question: """
 ${content}
 """

 Answer as markdown (including related code snippets if available) and use same language as question language event if you dont know:
 `;

 const complete = await openai.completions.create({
 model: 'gpt-3.5-turbo-instruct',
 prompt,
 max_tokens: 1500,
 temperature: 0,
 stream: true
 });

 const stream = new ReadableStream({
 async start(controller) {
 for await (const part of complete) {
 const line = part.choices[0]?.text || '';
 controller.enqueue(line);
 }
 controller.close();
 },
 cancel() {
 console.log('abort stream');
 complete.controller.abort();
 }
 });

 return new Response(stream, {
 headers: {
 'Content-Type': 'text/event-stream'
 }
 });
};

```

And that concludes the overview. Delving into detailed code explanations would exceed the scope of a single post. However, if you’re keen on exploring the nitty-gritty details, feel free to explore the GitHub repository: [Knowledge AI](https://github.com/suciptoid/knowledge-ai). Also, check out this snapshot of my interaction with the notes:

![Knowledge AI](https://sucipto.id/images/uploads/knowledge-ai.png)

Happy coding!