Repository: giselles-ai/giselle
Pull request #812: Add GitHub Vector Store (Blobs only)
Author: login: satococoa
Created: 2025-05-08T08:16:37Z | Merged: 2025-05-15T01:04:07Z

> [!note]
> migrations have been already applied in #832 

## Summary
Added functionality to import GitHub repository code into Vector Store.
This allows teams to use GitHub code as a searchable Vector Store.

The management screen enables repository registration and deletion, and we've implemented a mechanism for periodic data ingestion using Vercel Cron.

Because the Vector Store feature is gated behind a feature flag, it is hidden from regular users.

> [!tip]
> The initial ingestion of the *giselle* repository took **11 minutes**.


## Changes
### studio.giselles.ai
* Added `/settings/team/vector-stores` page
   * Implemented Vector Store registration and deletion functionality
* Added `/api/vector-stores/github/ingest` API endpoint
   * Configured Vercel Cron to run automatically every 10 minutes
   * Note: Vercel Cron does not work in Preview environments, so manual execution is required
   * Manual execution example: `curl http://localhost:3000/api/vector-stores/github/ingest -H 'authorization: Bearer <CRON_SECRET>'`


### packages/rag
* `ingest()` orchestrator: load → chunk → embed -> store

### `packages/github-tool`
* Added `GitHubBlobLoader`

## Future Work
* Implement Vector Store Node in Workspace
* Implement differential ingestion
* Introduce job queue system

## Screenshots
<img width="1066" alt="スクリーンショット 2025-05-13 10 39 13" src="https://github.com/user-attachments/assets/de4410f7-b68a-462f-b216-2e505b2f5232" />

<img width="1066" alt="スクリーンショット 2025-05-13 10 39 19" src="https://github.com/user-attachments/assets/9c48d550-9697-41c2-984b-266eff7d985a" />

<img width="1066" alt="スクリーンショット 2025-05-13 10 39 27" src="https://github.com/user-attachments/assets/8f2c73c0-9362-47ea-9b43-da1b5debbcd1" />

<img width="1066" alt="スクリーンショット 2025-05-13 10 39 42" src="https://github.com/user-attachments/assets/22f92ca6-f1bc-4b65-88f4-a39e35f0a873" />





Diff:
diff --git a/apps/studio.giselles.ai/app/(main)/layout.tsx b/apps/studio.giselles.ai/app/(main)/layout.tsx
index 28f321fbf5..df3288599f 100644
--- a/apps/studio.giselles.ai/app/(main)/layout.tsx
+++ b/apps/studio.giselles.ai/app/(main)/layout.tsx
@@ -1,4 +1,5 @@
 import { GiselleLogo } from "@/components/giselle-logo";
+import { githubVectorStoreFlag } from "@/flags";
 import { UserButton } from "@/services/accounts/components";
 import { TeamSelection } from "@/services/teams/components/team-selection";
 import Link from "next/link";
@@ -6,6 +7,7 @@ import type { ReactNode } from "react";
 import { Nav } from "./nav";
 
 export default async function Layout({ children }: { children: ReactNode }) {
+	const githubVectorStore = await githubVectorStoreFlag();
 	return (
 		<div className="h-screen overflow-y-hidden bg-black-900 flex flex-col">
 			<header className="flex flex-col">
@@ -31,8 +33,7 @@ export default async function Layout({ children }: { children: ReactNode }) {
 					</div>
 				</div>
 
-				{/* 下の行: メインナビゲーション */}
-				<Nav />
+				<Nav githubVectorStoreFlag={githubVectorStore} />
 
 				{/* Border line below navigation */}
 				<div className="h-[1px] w-full bg-black-800" />
diff --git a/apps/studio.giselles.ai/app/(main)/nav.tsx b/apps/studio.giselles.ai/app/(main)/nav.tsx
index 0ecb73f3be..1b49ecffb5 100644
--- a/apps/studio.giselles.ai/app/(main)/nav.tsx
+++ b/apps/studio.giselles.ai/app/(main)/nav.tsx
@@ -9,11 +9,17 @@ const menuItems = [
 	{ name: "Apps", path: "/apps" },
 	{ name: "Members", path: "/settings/team/members" },
 	{ name: "Integrations", path: "/settings/team/integrations" },
+	{ name: "Vector Stores", path: "/settings/team/vector-stores" },
 	{ name: "Usage", path: "/settings/team/usage" },
 	{ name: "Team Settings", path: "/settings/team" },
 ];
 
-export const Nav: FC = () => {
+// receive githubVectorStoreFlag as props
+interface NavProps {
+	githubVectorStoreFlag: boolean;
+}
+
+export const Nav: FC<NavProps> = ({ githubVectorStoreFlag }) => {
 	const pathname = usePathname();
 
 	// hide nav on settings/account page
@@ -21,11 +27,16 @@ export const Nav: FC = () => {
 		return null;
 	}
 
-	// 現在のパスに最もマッチする項目を見つける
+	// remove Vector Store link
+	const filteredMenuItems = githubVectorStoreFlag
+		? menuItems
+		: menuItems.filter((item) => item.name !== "Vector Stores");
+
+	// find the best match path
 	let bestMatchPath = "";
 	let bestMatchIndex = -1;
 
-	menuItems.forEach((item, index) => {
+	filteredMenuItems.forEach((item, index) => {
 		if (
 			pathname.startsWith(item.path) &&
 			item.path.length > bestMatchPath.length
@@ -38,7 +49,7 @@ export const Nav: FC = () => {
 	return (
 		<div className="flex items-center px-[24px] py-0 border-t border-black-900/50">
 			<div className="flex items-center">
-				{menuItems.map((item, index) => {
+				{filteredMenuItems.map((item, index) => {
 					const isActive = index === bestMatchIndex;
 
 					return (
diff --git a/apps/studio.giselles.ai/app/(main)/settings/integration/page.tsx b/apps/studio.giselles.ai/app/(main)/settings/integration/page.tsx
deleted file mode 100644
index 36b11d3e00..0000000000
--- a/apps/studio.giselles.ai/app/(main)/settings/integration/page.tsx
+++ /dev/null
@@ -1,15 +0,0 @@
-import { GitHubIntegration } from "./github-integration";
-
-export default async function IntegrationPage() {
-	return (
-		<div className="grid gap-[16px]">
-			<h3
-				className="text-[32px] text-black--30 font-rosart"
-				style={{ textShadow: "0px 0px 20px hsla(207, 100%, 48%, 1)" }}
-			>
-				Integrations
-			</h3>
-			<GitHubIntegration />
-		</div>
-	);
-}
diff --git a/apps/studio.giselles.ai/app/(main)/settings/integration/github-integration.tsx b/apps/studio.giselles.ai/app/(main)/settings/team/integrations/github-integration.tsx
similarity index 98%
rename from apps/studio.giselles.ai/app/(main)/settings/integration/github-integration.tsx
rename to apps/studio.giselles.ai/app/(main)/settings/team/integrations/github-integration.tsx
index 6be0f6f7f1..8a9d13ae01 100644
--- a/apps/studio.giselles.ai/app/(main)/settings/integration/github-integration.tsx
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/integrations/github-integration.tsx
@@ -4,7 +4,7 @@ import { gitHubAppInstallURL } from "@/services/external/github";
 import { SiGithub } from "@icons-pack/react-simple-icons";
 import type { components } from "@octokit/openapi-types";
 import Link from "next/link";
-import { Button } from "../components/button";
+import { Button } from "../../components/button";
 
 export async function GitHubIntegration() {
 	const installUrl = await gitHubAppInstallURL();
diff --git a/apps/studio.giselles.ai/app/(main)/settings/team/integrations/page.tsx b/apps/studio.giselles.ai/app/(main)/settings/team/integrations/page.tsx
index b5e944012c..43f6f876c4 100644
--- a/apps/studio.giselles.ai/app/(main)/settings/team/integrations/page.tsx
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/integrations/page.tsx
@@ -1,4 +1,4 @@
-import { GitHubIntegration } from "../../integration/github-integration";
+import { GitHubIntegration } from "./github-integration";
 
 export default async function TeamIntegrationsPage() {
 	return (
diff --git a/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
new file mode 100644
index 0000000000..13dbdcdc16
--- /dev/null
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
@@ -0,0 +1,72 @@
+"use server";
+
+import { db, githubRepositoryIndex } from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { buildAppInstallationClient } from "@/services/external/github";
+import { fetchCurrentTeam } from "@/services/teams";
+import { createId } from "@paralleldrive/cuid2";
+import { and, eq } from "drizzle-orm";
+import { revalidatePath } from "next/cache";
+
+export async function registerRepositoryIndex(
+	owner: string,
+	repo: string,
+	installationId: number,
+) {
+	const team = await fetchCurrentTeam();
+
+	// check if the user have access to the installation
+	// FIXME: When the installation is managed by the team, we should use the team's installation instead
+	const githubIdentityState = await getGitHubIdentityState();
+	if (githubIdentityState.status !== "authorized") {
+		throw new Error("User is not authorized to access the repository");
+	}
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	const installation = installationData.installations.find(
+		(installation) => installation.id === installationId,
+	);
+	if (!installation) {
+		throw new Error("Installation not found");
+	}
+
+	// check if the installation can access the repository
+	const installationClient = await buildAppInstallationClient(installationId);
+
+	// check if the repository is already registered
+	const repository = await installationClient.request(
+		"GET /repos/{owner}/{repo}",
+		{
+			owner,
+			repo,
+		},
+	);
+	if (repository.status !== 200) {
+		throw new Error("Repository not found");
+	}
+
+	const newIndexId = `gthbi_${createId()}` as GitHubRepositoryIndexId;
+	await db.insert(githubRepositoryIndex).values({
+		id: newIndexId,
+		owner,
+		repo,
+		teamDbId: team.dbId,
+		installationId,
+	});
+
+	revalidatePath("/settings/team/vector-stores");
+}
+
+export async function deleteRepositoryIndex(indexId: GitHubRepositoryIndexId) {
+	const team = await fetchCurrentTeam();
+	await db
+		.delete(githubRepositoryIndex)
+		.where(
+			and(
+				eq(githubRepositoryIndex.teamDbId, team.dbId),
+				eq(githubRepositoryIndex.id, indexId),
+			),
+		);
+	revalidatePath("/settings/team/vector-stores");
+}
diff --git a/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/page.tsx b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/page.tsx
new file mode 100644
index 0000000000..e817dfe25e
--- /dev/null
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/page.tsx
@@ -0,0 +1,216 @@
+import { Card } from "@/components/ui/card";
+import { db, githubRepositoryIndex } from "@/drizzle";
+import { githubVectorStoreFlag } from "@/flags";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { fetchCurrentTeam } from "@/services/teams";
+import { desc, eq } from "drizzle-orm";
+import { ExternalLink } from "lucide-react";
+import { notFound } from "next/navigation";
+import { deleteRepositoryIndex, registerRepositoryIndex } from "./actions";
+import { RepositoryItem } from "./repository-item";
+import { RepositoryRegistrationDialog } from "./repository-registration-dialog";
+
+export default async function TeamVectorStorePage() {
+	const vectorStoreFlag = await githubVectorStoreFlag();
+	if (!vectorStoreFlag) {
+		return notFound();
+	}
+
+	const githubIdentityState = await getGitHubIdentityState();
+	if (
+		githubIdentityState.status === "unauthorized" ||
+		githubIdentityState.status === "invalid-credential"
+	) {
+		return <GitHubAuthRequiredCard />;
+	}
+
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	if (installationData.total_count === 0) {
+		return <GitHubAppInstallRequiredCard />;
+	}
+	const installations = installationData.installations;
+
+	const installationsWithRepos = await Promise.all(
+		installations.map(async (installation) => {
+			const repos = await userClient.getRepositories(installation.id);
+			const installationId = installation.id;
+			if (!installation.account) {
+				throw new Error("Installation account is null");
+			}
+
+			const installationName =
+				"login" in installation.account
+					? installation.account.login
+					: installation.account.name;
+
+			return {
+				installation: {
+					id: installationId,
+					name: installationName,
+				},
+				repositories: repos.repositories.map((repo) => ({
+					id: repo.id,
+					owner: repo.owner.login,
+					name: repo.name,
+				})),
+			};
+		}),
+	);
+
+	const repositoryIndexes = await getGitHubRepositoryIndexes();
+
+	return (
+		<div className="flex flex-col gap-[24px]">
+			<div className="flex justify-between items-center">
+				<h3
+					className="text-primary-100 font-semibold text-[28px] leading-[28px] tracking-[-0.011em] font-hubot"
+					style={{ textShadow: "0px 0px 20px hsla(207, 100%, 48%, 1)" }}
+				>
+					Vector Stores
+				</h3>
+				<a
+					href="https://docs.giselles.ai/guides/settings/team/vector-store"
+					target="_blank"
+					rel="noopener noreferrer"
+					className="text-black-300 text-[14px] font-medium border border-black-300 rounded-[4px] px-1.5 py-0.5 hover:bg-black-300/10 flex items-center gap-1.5 font-hubot"
+				>
+					About Vector Stores
+					<ExternalLink size={14} />
+				</a>
+			</div>
+
+			<RepositoryListCard
+				registrationDialog={
+					<RepositoryRegistrationDialog
+						installationsWithRepos={installationsWithRepos}
+						registerRepositoryIndexAction={registerRepositoryIndex}
+					/>
+				}
+				repositoryIndexes={repositoryIndexes}
+			/>
+		</div>
+	);
+}
+
+function GitHubAuthRequiredCard() {
+	return (
+		<div className="flex flex-col gap-[24px]">
+			<h3
+				className="text-primary-100 font-semibold text-[28px] leading-[28px] tracking-[-0.011em] font-hubot"
+				style={{ textShadow: "0px 0px 20px hsla(207, 100%, 48%, 1)" }}
+			>
+				Vector Store
+			</h3>
+			<Card className="border-[0.5px] border-black-400 rounded-[8px] bg-transparent p-6">
+				<div className="flex flex-col items-center justify-center py-8">
+					<h4 className="text-white-400 font-medium text-[18px] leading-[21.6px] font-hubot mb-2">
+						You need to authenticate your GitHub account.
+					</h4>
+					<p className="text-black-400 text-[14px] leading-[20.4px] font-geist text-center mb-4">
+						To use Vector Store, you need to authenticate your GitHub account.
+						Please authenticate in the account settings.
+					</p>
+					<a
+						href="/settings/account/authentication"
+						className="text-black-300 text-[14px] font-medium border border-black-300 rounded-[4px] px-4 py-2 hover:bg-black-300/10 flex items-center gap-1.5 font-hubot"
+					>
+						Open Authentication Settings
+					</a>
+				</div>
+			</Card>
+		</div>
+	);
+}
+
+function GitHubAppInstallRequiredCard() {
+	return (
+		<div className="flex flex-col gap-[24px]">
+			<h3
+				className="text-primary-100 font-semibold text-[28px] leading-[28px] tracking-[-0.011em] font-hubot"
+				style={{ textShadow: "0px 0px 20px hsla(207, 100%, 48%, 1)" }}
+			>
+				Vector Store
+			</h3>
+			<Card className="border-[0.5px] border-black-400 rounded-[8px] bg-transparent p-6">
+				<div className="flex flex-col items-center justify-center py-8">
+					<h4 className="text-white-400 font-medium text-[18px] leading-[21.6px] font-hubot mb-2">
+						You need to install Giselle's GitHub App.
+					</h4>
+					<p className="text-black-400 text-[14px] leading-[20.4px] font-geist text-center mb-4">
+						To use Vector Store, you need to install Giselle's GitHub App.
+						Please install in the integrations settings.
+					</p>
+					<a
+						href="/settings/team/integrations"
+						className="text-black-300 text-[14px] font-medium border border-black-300 rounded-[4px] px-4 py-2 hover:bg-black-300/10 flex items-center gap-1.5 font-hubot"
+					>
+						Open Integrations Settings
+					</a>
+				</div>
+			</Card>
+		</div>
+	);
+}
+
+type RepositoryListCardProps = {
+	registrationDialog: React.ReactNode;
+	repositoryIndexes: Awaited<ReturnType<typeof getGitHubRepositoryIndexes>>;
+};
+
+function RepositoryListCard({
+	registrationDialog,
+	repositoryIndexes,
+}: RepositoryListCardProps) {
+	return (
+		<div className="flex flex-col gap-y-[16px]">
+			<Card className="border-[0.5px] border-black-400 rounded-[8px] bg-transparent p-6">
+				<div className="flex items-center mb-4">
+					<div>
+						<h4 className="text-white-400 font-medium text-[18px] leading-[21.6px] font-hubot">
+							GitHub Repositories
+						</h4>
+						<p className="text-black-400 text-[14px] leading-[20.4px] font-geist mt-1">
+							You can ingest your project's code into Vector Store and use it in
+							GitHub Vector Store Nodes.
+						</p>
+					</div>
+					<div className="ml-auto">{registrationDialog}</div>
+				</div>
+
+				{repositoryIndexes.length > 0 ? (
+					<div className="space-y-4">
+						{repositoryIndexes.map((index) => (
+							<RepositoryItem
+								key={index.id}
+								repositoryIndex={index}
+								deleteRepositoryIndexAction={deleteRepositoryIndex}
+							/>
+						))}
+					</div>
+				) : (
+					<EmptyRepositoryCard />
+				)}
+			</Card>
+		</div>
+	);
+}
+
+function EmptyRepositoryCard() {
+	return (
+		<div className="text-black-400 text-center py-8">
+			No repositories are registered. Please register a repository using the
+			"Register Repository" button.
+		</div>
+	);
+}
+
+async function getGitHubRepositoryIndexes() {
+	const team = await fetchCurrentTeam();
+	const records = await db
+		.select()
+		.from(githubRepositoryIndex)
+		.where(eq(githubRepositoryIndex.teamDbId, team.dbId))
+		.orderBy(desc(githubRepositoryIndex.dbId));
+	return records;
+}
diff --git a/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-item.tsx b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-item.tsx
new file mode 100644
index 0000000000..405ae327c4
--- /dev/null
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-item.tsx
@@ -0,0 +1,154 @@
+"use client";
+
+import {
+	AlertDialog,
+	AlertDialogAction,
+	AlertDialogCancel,
+	AlertDialogContent,
+	AlertDialogDescription,
+	AlertDialogFooter,
+	AlertDialogHeader,
+	AlertDialogTitle,
+	AlertDialogTrigger,
+} from "@/components/ui/alert-dialog";
+import { Button } from "@/components/ui/button";
+import type {
+	GitHubRepositoryIndexStatus,
+	githubRepositoryIndex,
+} from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { Trash } from "lucide-react";
+import { useState, useTransition } from "react";
+
+type RepositoryItemProps = {
+	repositoryIndex: typeof githubRepositoryIndex.$inferSelect;
+	deleteRepositoryIndexAction: (
+		indexId: GitHubRepositoryIndexId,
+	) => Promise<void>;
+};
+
+export function RepositoryItem({
+	repositoryIndex,
+	deleteRepositoryIndexAction,
+}: RepositoryItemProps) {
+	const [showDeleteDialog, setShowDeleteDialog] = useState(false);
+	const [isPending, startTransition] = useTransition();
+
+	const handleDelete = async () => {
+		startTransition(async () => {
+			try {
+				await deleteRepositoryIndexAction(repositoryIndex.id);
+				setShowDeleteDialog(false);
+			} catch (error) {
+				console.error(error);
+			}
+		});
+	};
+
+	return (
+		<div className="border border-black-400 rounded-md p-4 bg-black-900/50">
+			<div className="flex justify-between items-center">
+				<div>
+					<h5 className="text-white-400 font-medium text-[16px] leading-[19.2px] font-hubot">
+						{repositoryIndex.owner}/{repositoryIndex.repo}
+					</h5>
+					<div className="flex items-center gap-2 mt-1">
+						<StatusBadge status={repositoryIndex.status} />
+						{repositoryIndex.lastIngestedCommitSha && (
+							<span className="text-black-400 text-[12px] leading-[20.4px] font-geist">
+								Last Ingested:{" "}
+								{repositoryIndex.lastIngestedCommitSha.substring(0, 7)}
+							</span>
+						)}
+					</div>
+				</div>
+				<div className="ml-auto flex gap-2 items-center">
+					<AlertDialog
+						open={showDeleteDialog}
+						onOpenChange={setShowDeleteDialog}
+					>
+						<AlertDialogTrigger asChild>
+							<Button
+								variant="destructive"
+								className="h-8 px-3 text-[12px] flex items-center gap-1 rounded-md transition-colors duration-150"
+								disabled={isPending}
+								onClick={() => setShowDeleteDialog(true)}
+							>
+								<Trash className="h-4 w-4 mr-1" />
+								{isPending ? "Deleting..." : "Delete"}
+							</Button>
+						</AlertDialogTrigger>
+						<AlertDialogContent>
+							<AlertDialogHeader>
+								<AlertDialogTitle>Delete Repository</AlertDialogTitle>
+								<AlertDialogDescription>
+									This action cannot be undone. Are you sure you want to delete
+									this repository?
+								</AlertDialogDescription>
+							</AlertDialogHeader>
+							<AlertDialogFooter>
+								<AlertDialogCancel
+									type="button"
+									onClick={() => setShowDeleteDialog(false)}
+									disabled={isPending}
+									className="py-2 px-4 border-[0.5px] border-black-400 rounded-[8px] font-hubot"
+								>
+									Cancel
+								</AlertDialogCancel>
+								<AlertDialogAction
+									type="submit"
+									onClick={handleDelete}
+									disabled={isPending}
+									className="py-2 px-4 bg-error-900 rounded-[8px] text-white-400 font-hubot"
+								>
+									{isPending ? "Deleting..." : "Delete"}
+								</AlertDialogAction>
+							</AlertDialogFooter>
+						</AlertDialogContent>
+					</AlertDialog>
+				</div>
+			</div>
+		</div>
+	);
+}
+
+function StatusBadge({ status }: { status: GitHubRepositoryIndexStatus }) {
+	let bgColor = "bg-gray-500";
+	let textColor = "text-white";
+	let label = "Unknown";
+
+	switch (status) {
+		case "idle":
+			bgColor = "bg-gray-500";
+			textColor = "text-white";
+			label = "Waiting for ingestion";
+			break;
+		case "running":
+			bgColor = "bg-blue-500";
+			textColor = "text-white";
+			label = "Ingesting";
+			break;
+		case "completed":
+			bgColor = "bg-green-500";
+			textColor = "text-white";
+			label = "Ready";
+			break;
+		case "failed":
+			bgColor = "bg-red-500";
+			textColor = "text-white";
+			label = "Error";
+			break;
+		default: {
+			const _exhaustiveCheck: never = status;
+			throw new Error(`Unknown status: ${_exhaustiveCheck}`);
+		}
+	}
+
+	return (
+		<span
+			className={`${bgColor} ${textColor} text-[10px] leading-[12px] font-semibold px-2 py-0.5 rounded-full`}
+		>
+			{label}
+		</span>
+	);
+}
diff --git a/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-registration-dialog.tsx b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-registration-dialog.tsx
new file mode 100644
index 0000000000..e65762e5bf
--- /dev/null
+++ b/apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/repository-registration-dialog.tsx
@@ -0,0 +1,239 @@
+"use client";
+
+import {
+	Dialog,
+	DialogContent,
+	DialogHeader,
+	DialogTitle,
+	DialogTrigger,
+} from "@/components/ui/dialog";
+import {
+	Select,
+	SelectContent,
+	SelectItem,
+	SelectTrigger,
+	SelectValue,
+} from "@/components/ui/select";
+import { useState, useTransition } from "react";
+import { Button } from "../../components/button";
+
+type Installation = {
+	id: number;
+	name: string;
+};
+
+type Repository = {
+	id: number;
+	owner: string;
+	name: string;
+};
+
+export type InstallationWithRepos = {
+	installation: Installation;
+	repositories: Repository[];
+};
+
+type RepositoryRegistrationDialogProps = {
+	installationsWithRepos: InstallationWithRepos[];
+	registerRepositoryIndexAction: (
+		owner: string,
+		repo: string,
+		installationId: number,
+	) => Promise<void>;
+};
+
+export function RepositoryRegistrationDialog({
+	installationsWithRepos,
+	registerRepositoryIndexAction,
+}: RepositoryRegistrationDialogProps) {
+	const [isOpen, setIsOpen] = useState(false);
+	const [ownerId, setOwnerId] = useState<string>("");
+	const [repositoryId, setRepositoryId] = useState<string>("");
+	const [error, setError] = useState("");
+	const [isPending, startTransition] = useTransition();
+
+	const selectedInstallation = installationsWithRepos.find(
+		(i) => String(i.installation.id) === ownerId,
+	);
+	const repositoryOptions = selectedInstallation?.repositories || [];
+
+	const handleSubmit = async (e: React.FormEvent) => {
+		e.preventDefault();
+		setError("");
+
+		if (!ownerId) {
+			setError("Owner is required");
+			return;
+		}
+		if (!repositoryId) {
+			setError("Repository is required");
+			return;
+		}
+
+		startTransition(async () => {
+			try {
+				const owner = installationsWithRepos.find(
+					(i) => String(i.installation.id) === ownerId,
+				)?.installation.name;
+				if (!owner) {
+					setError("Owner not found");
+					return;
+				}
+				const repo = repositoryOptions.find(
+					(r) => String(r.id) === repositoryId,
+				)?.name;
+				if (!repo) {
+					setError("Repository not found");
+					return;
+				}
+				await registerRepositoryIndexAction(owner, repo, Number(ownerId));
+				setIsOpen(false);
+				setOwnerId("");
+				setRepositoryId("");
+			} catch (error) {
+				setError("An error occurred");
+				console.error(error);
+			}
+		});
+	};
+
+	return (
+		<Dialog open={isOpen} onOpenChange={setIsOpen}>
+			<DialogTrigger asChild>
+				<Button variant="default">Register Repository</Button>
+			</DialogTrigger>
+			<DialogContent
+				className="gap-y-6 px-[57px] py-[40px] max-w-[380px] w-full bg-black-900 border-none rounded-[16px] bg-linear-to-br/hsl from-black-600 to-black-250 sm:rounded-[16px]"
+				style={{
+					animation: "fadeIn 0.2s ease-out",
+					transformOrigin: "center",
+				}}
+			>
+				<style jsx global>{`
+					@keyframes fadeIn {
+						from {
+							opacity: 0;
+							transform: scale(0.95);
+						}
+						to {
+							opacity: 1;
+							transform: scale(1);
+						}
+					}
+				`}</style>
+				<div
+					aria-hidden="true"
+					className="absolute inset-0 rounded-[16px] border-[0.5px] border-transparent bg-black-900 bg-clip-padding"
+				/>
+				<DialogHeader className="relative z-10">
+					<DialogTitle className="text-white-800 font-semibold text-[20px] leading-[28px] font-hubot text-center">
+						Register GitHub Repository
+					</DialogTitle>
+				</DialogHeader>
+				<form
+					onSubmit={handleSubmit}
+					className="flex flex-col gap-y-4 relative z-10"
+				>
+					<div className="flex flex-col gap-y-2">
+						<label
+							htmlFor="owner"
+							className="text-white-400 text-[14px] leading-[16.8px] font-hubot"
+						>
+							Owner
+						</label>
+						<Select
+							value={ownerId}
+							onValueChange={(val) => {
+								setOwnerId(val);
+								setRepositoryId("");
+							}}
+							disabled={isPending}
+						>
+							<SelectTrigger
+								id="owner"
+								className="py-2 rounded-[8px] w-full bg-white-30/30 text-black-800 font-medium text-[12px] leading-[20.4px] font-geist shadow-none focus:text-white border border-white-400"
+							>
+								<SelectValue placeholder="Select owner" />
+							</SelectTrigger>
+							<SelectContent>
+								{installationsWithRepos.length === 0 ? (
+									<div className="px-3 py-2 text-muted-foreground text-sm">
+										No owners available
+									</div>
+								) : (
+									installationsWithRepos.map(({ installation }) => (
+										<SelectItem
+											key={installation.id}
+											value={String(installation.id)}
+										>
+											{installation.name}
+										</SelectItem>
+									))
+								)}
+							</SelectContent>
+						</Select>
+					</div>
+					<div className="flex flex-col gap-y-2">
+						<label
+							htmlFor="repo"
+							className="text-white-400 text-[14px] leading-[16.8px] font-hubot"
+						>
+							Repository Name
+						</label>
+						<Select
+							value={repositoryId}
+							onValueChange={setRepositoryId}
+							disabled={isPending || !ownerId}
+						>
+							<SelectTrigger
+								id="repo"
+								className="py-2 rounded-[8px] w-full bg-white-30/30 text-black-800 font-medium text-[12px] leading-[20.4px] font-geist shadow-none focus:text-white border border-white-400"
+							>
+								<SelectValue placeholder="Select repository" />
+							</SelectTrigger>
+							<SelectContent>
+								{!ownerId ? (
+									<div className="px-3 py-2 text-muted-foreground text-sm">
+										Select owner first
+									</div>
+								) : repositoryOptions.length === 0 ? (
+									<div className="px-3 py-2 text-muted-foreground text-sm">
+										No repositories available
+									</div>
+								) : (
+									repositoryOptions.map((repo) => (
+										<SelectItem key={repo.id} value={String(repo.id)}>
+											{repo.name}
+										</SelectItem>
+									))
+								)}
+							</SelectContent>
+						</Select>
+					</div>
+					{error && (
+						<p className="text-[12px] leading-[20.4px] text-error-900 font-geist">
+							{error}
+						</p>
+					)}
+					<div className="flex justify-end space-x-4 mt-2">
+						<Button
+							type="button"
+							onClick={() => setIsOpen(false)}
+							disabled={isPending}
+							className="w-full h-[38px] bg-transparent border-black-400 text-black-400 text-[16px] leading-[19.2px] tracking-[-0.04em] hover:bg-transparent hover:text-black-400 "
+						>
+							Cancel
+						</Button>
+						<Button
+							type="submit"
+							disabled={isPending}
+							className="w-full h-[38px] text-[16px] leading-[19.2px] tracking-[-0.04em]"
+						>
+							Register
+						</Button>
+					</div>
+				</form>
+			</DialogContent>
+		</Dialog>
+	);
+}
diff --git a/apps/studio.giselles.ai/app/api/vector-stores/github/ingest/route.ts b/apps/studio.giselles.ai/app/api/vector-stores/github/ingest/route.ts
new file mode 100644
index 0000000000..f4ad10f648
--- /dev/null
+++ b/apps/studio.giselles.ai/app/api/vector-stores/github/ingest/route.ts
@@ -0,0 +1,351 @@
+import {
+	db,
+	githubRepositoryEmbeddings,
+	githubRepositoryIndex,
+} from "@/drizzle";
+import {
+	GitHubBlobLoader,
+	type GitHubBlobMetadata,
+	type GithubRepositoryParams,
+	octokit,
+} from "@giselle-sdk/github-tool";
+import {
+	type BaseEmbedding,
+	type EmbeddingStore,
+	ingest,
+} from "@giselle-sdk/rag";
+import { captureException } from "@sentry/nextjs";
+import { and, eq } from "drizzle-orm";
+import type { NextRequest } from "next/server";
+
+export const maxDuration = 800;
+
+/**
+ * GitHub repository embedding data structure
+ */
+interface GitHubRepositoryEmbedding {
+	owner: string;
+	repo: string;
+	commitSha: string;
+	fileSha: string;
+	path: string;
+	nodeId: string;
+	chunkIndex: number;
+	chunkContent: string;
+	embedding: number[];
+}
+
+// ingest GitHub Code
+export async function GET(request: NextRequest) {
+	const authHeader = request.headers.get("authorization");
+	if (authHeader !== `Bearer ${process.env.CRON_SECRET}`) {
+		return new Response("Unauthorized", {
+			status: 401,
+		});
+	}
+
+	const targetGitHubRepositories = await fetchTargetGitHubRepositories();
+
+	for (const targetGitHubRepository of targetGitHubRepositories) {
+		const { owner, repo, installationId, lastIngestedCommitSha, teamDbId } =
+			targetGitHubRepository;
+
+		const appId = process.env.GITHUB_APP_ID;
+		if (!appId) {
+			throw new Error("GITHUB_APP_ID is empty");
+		}
+		const privateKey = process.env.GITHUB_APP_PRIVATE_KEY;
+		if (!privateKey) {
+			throw new Error("GITHUB_APP_PRIVATE_KEY is empty");
+		}
+
+		const octokitClient = octokit({
+			strategy: "app-installation",
+			appId,
+			privateKey,
+			installationId,
+		});
+
+		// Create GitHub repository loader
+		const loader = new GitHubBlobLoader(octokitClient, {
+			maxBlobSize: 1 * 1024 * 1024, // 1MB limit
+		});
+
+		const source: GithubRepositoryParams = {
+			owner,
+			repo,
+		};
+
+		const embeddingStore = new GitHubRepositoryEmbeddingStoreImpl(teamDbId);
+
+		// Ingest using the RAG package
+		await ingest({
+			source,
+			loader,
+			store: embeddingStore,
+			transformEmbedding: transformGitHubEmbedding,
+		});
+	}
+
+	return new Response("ok", { status: 200 });
+}
+
+/**
+ * Transform base embedding to GitHub repository embedding
+ */
+function transformGitHubEmbedding(
+	baseEmbedding: BaseEmbedding,
+	metadata: GitHubBlobMetadata,
+): GitHubRepositoryEmbedding {
+	return {
+		owner: metadata.owner,
+		repo: metadata.repo,
+		commitSha: metadata.commitSha,
+		fileSha: metadata.fileSha,
+		path: metadata.path,
+		nodeId: metadata.nodeId,
+		chunkIndex: baseEmbedding.chunkIndex,
+		chunkContent: baseEmbedding.chunkContent,
+		embedding: baseEmbedding.embedding,
+	};
+}
+
+type TargetGitHubRepository = {
+	owner: string;
+	repo: string;
+	teamDbId: number;
+	installationId: number;
+	lastIngestedCommitSha: string | null;
+};
+
+async function fetchTargetGitHubRepositories(): Promise<
+	TargetGitHubRepository[]
+> {
+	const records = await db
+		.select({
+			owner: githubRepositoryIndex.owner,
+			repo: githubRepositoryIndex.repo,
+			installationId: githubRepositoryIndex.installationId,
+			lastIngestedCommitSha: githubRepositoryIndex.lastIngestedCommitSha,
+			teamDbId: githubRepositoryIndex.teamDbId,
+		})
+		.from(githubRepositoryIndex)
+		.where(eq(githubRepositoryIndex.status, "idle"));
+
+	return records.map((record) => ({
+		owner: record.owner,
+		repo: record.repo,
+		installationId: record.installationId,
+		lastIngestedCommitSha: record.lastIngestedCommitSha,
+		teamDbId: record.teamDbId,
+	}));
+}
+
+/**
+ * Implementation of EmbeddingStore for GitHub repositories
+ */
+class GitHubRepositoryEmbeddingStoreImpl
+	implements EmbeddingStore<GitHubRepositoryEmbedding>
+{
+	private teamDbId: number;
+	constructor(teamDbId: number) {
+		this.teamDbId = teamDbId;
+	}
+
+	private async getRepositoryIndexDbId(owner: string, repo: string) {
+		return this.withPgRetry(async () => {
+			const records = await db
+				.select({ dbId: githubRepositoryIndex.dbId })
+				.from(githubRepositoryIndex)
+				.where(
+					and(
+						eq(githubRepositoryIndex.owner, owner),
+						eq(githubRepositoryIndex.repo, repo),
+						eq(githubRepositoryIndex.teamDbId, this.teamDbId),
+					),
+				)
+				.limit(1);
+			const repositoryIndex = records[0];
+			if (repositoryIndex == null) {
+				throw new Error(`Repository index not found: ${owner}/${repo}`);
+			}
+			return repositoryIndex.dbId;
+		});
+	}
+
+	async insertEmbedding(data: GitHubRepositoryEmbedding): Promise<void> {
+		const repositoryIndexDbId = await this.getRepositoryIndexDbId(
+			data.owner,
+			data.repo,
+		);
+		await this.withPgRetry(async () => {
+			await db
+				.insert(githubRepositoryEmbeddings)
+				.values({
+					repositoryIndexDbId,
+					commitSha: data.commitSha,
+					fileSha: data.fileSha,
+					path: data.path,
+					nodeId: data.nodeId,
+					embedding: data.embedding,
+					chunkIndex: data.chunkIndex,
+					chunkContent: data.chunkContent,
+				})
+				.onConflictDoNothing();
+		});
+	}
+
+	async deleteEmbedding(
+		key: Partial<GitHubRepositoryEmbedding>,
+	): Promise<void> {
+		if (key.owner != null && key.repo != null && key.path != null) {
+			const { owner, repo, path } = key;
+
+			const repositoryIndexDbId = await this.getRepositoryIndexDbId(
+				owner,
+				repo,
+			);
+			await this.withPgRetry(async () => {
+				await db
+					.delete(githubRepositoryEmbeddings)
+					.where(
+						and(
+							eq(
+								githubRepositoryEmbeddings.repositoryIndexDbId,
+								repositoryIndexDbId,
+							),
+							eq(githubRepositoryEmbeddings.path, path),
+						),
+					);
+			});
+		}
+	}
+
+	async updateEmbedding(data: GitHubRepositoryEmbedding): Promise<void> {
+		// retry is done in deleteEmbedding and insertEmbedding
+		await this.deleteEmbedding(data);
+		await this.insertEmbedding(data);
+	}
+
+	async startIngestion(params: Record<string, unknown>): Promise<void> {
+		const source = params.source as GithubRepositoryParams;
+		if (!source) {
+			console.warn("startIngestion: source is missing", params);
+			return;
+		}
+
+		await this.withPgRetry(async () => {
+			await db
+				.update(githubRepositoryIndex)
+				.set({ status: "running" })
+				.where(
+					and(
+						eq(githubRepositoryIndex.owner, source.owner),
+						eq(githubRepositoryIndex.repo, source.repo),
+					),
+				);
+		});
+	}
+
+	async completeIngestion(params: Record<string, unknown>): Promise<void> {
+		const source = params.source as GithubRepositoryParams;
+		if (!source) {
+			console.warn("completeIngestion: source is missing", params);
+			return;
+		}
+
+		const commitSha = params.commitSha as string;
+		await this.withPgRetry(async () => {
+			await db
+				.update(githubRepositoryIndex)
+				.set({
+					status: "completed",
+					lastIngestedCommitSha: commitSha,
+				})
+				.where(
+					and(
+						eq(githubRepositoryIndex.owner, source.owner),
+						eq(githubRepositoryIndex.repo, source.repo),
+					),
+				);
+		});
+	}
+
+	async failIngestion(
+		params: Record<string, unknown>,
+		error: Error,
+	): Promise<void> {
+		const source = params.source as GithubRepositoryParams;
+		if (!source) {
+			console.warn("failIngestion: source is missing", params);
+			captureException(error);
+			return;
+		}
+
+		captureException(error, {
+			extra: {
+				owner: source.owner,
+				repo: source.repo,
+			},
+		});
+		await this.withPgRetry(async () => {
+			await db
+				.update(githubRepositoryIndex)
+				.set({ status: "failed" })
+				.where(
+					and(
+						eq(githubRepositoryIndex.owner, source.owner),
+						eq(githubRepositoryIndex.repo, source.repo),
+					),
+				);
+		});
+	}
+
+	// ingesting job may be a long running job, so it would be better to care transient errors
+	// https://www.postgresql.org/docs/16/errcodes-appendix.html
+	private TRANSIENT_CODES = new Set([
+		// Class 08 - Connection Exception
+		"08000",
+		"08003",
+		"08006",
+		"08001",
+		"08004",
+		"08007",
+		// Class 53 - Resource Exception
+		"53300", // too_many_connections
+		// Class 40 - Transaction Rollback
+		"40001", // serialization_failure
+		"40P01", // deadlock_detected
+	]);
+
+	private isTransientError(error: unknown) {
+		if (
+			typeof error === "object" &&
+			error !== null &&
+			"code" in error &&
+			typeof error.code === "string"
+		) {
+			return this.TRANSIENT_CODES.has(error.code);
+		}
+		return false;
+	}
+
+	private async withPgRetry<T>(
+		fn: () => Promise<T>,
+		attempt = 0,
+		maxAttempts = 4,
+	): Promise<T> {
+		try {
+			return await fn();
+		} catch (error: unknown) {
+			if (!this.isTransientError(error)) {
+				throw error;
+			}
+			if (attempt >= maxAttempts) {
+				throw error;
+			}
+			await new Promise((r) => setTimeout(r, 100 * 2 ** attempt));
+			return this.withPgRetry(fn, attempt + 1, maxAttempts);
+		}
+	}
+}
diff --git a/apps/studio.giselles.ai/drizzle/schema.ts b/apps/studio.giselles.ai/drizzle/schema.ts
index d7d056b162..2d6bcf9e6e 100644
--- a/apps/studio.giselles.ai/drizzle/schema.ts
+++ b/apps/studio.giselles.ai/drizzle/schema.ts
@@ -271,7 +271,11 @@ export const invitations = pgTable(
 	}),
 );
 
-type GitHubRepositoryIndexStatus = "idle" | "running" | "completed" | "failed";
+export type GitHubRepositoryIndexStatus =
+	| "idle"
+	| "running"
+	| "completed"
+	| "failed";
 export const githubRepositoryIndex = pgTable(
 	"github_repository_index",
 	{
diff --git a/apps/studio.giselles.ai/flags.ts b/apps/studio.giselles.ai/flags.ts
index 170d7bf872..b74f389141 100644
--- a/apps/studio.giselles.ai/flags.ts
+++ b/apps/studio.giselles.ai/flags.ts
@@ -103,3 +103,16 @@ export const runV2Flag = flag<boolean>({
 		{ value: true, label: "Enable" },
 	],
 });
+
+export const githubVectorStoreFlag = flag<boolean>({
+	key: "github-vector-store",
+	async decide() {
+		return takeLocalEnv("GITHUB_VECTOR_STORE_FLAG");
+	},
+	description: "Enable GitHub Vector Store",
+	defaultValue: false,
+	options: [
+		{ value: false, label: "disable" },
+		{ value: true, label: "Enable" },
+	],
+});
diff --git a/apps/studio.giselles.ai/middleware.ts b/apps/studio.giselles.ai/middleware.ts
index 708e636310..9730888afc 100644
--- a/apps/studio.giselles.ai/middleware.ts
+++ b/apps/studio.giselles.ai/middleware.ts
@@ -21,6 +21,6 @@ export default supabaseMiddleware(async (user, request) => {
 
 export const config = {
 	matcher: [
-		"/((?!_next/static|_next/image|.well-known|dev|webhooks|legal|login|signup|join|pricing|password_reset|subscription|auth|api/cron|api/giselle|favicon.ico|.*\\.(?:svg|png|jpg|jpeg|gif|webp)$).*)",
+		"/((?!_next/static|_next/image|.well-known|dev|webhooks|legal|login|signup|join|pricing|password_reset|subscription|auth|api/giselle|api/vector-stores|favicon.ico|.*\\.(?:svg|png|jpg|jpeg|gif|webp)$).*)",
 	],
 };
diff --git a/apps/studio.giselles.ai/package.json b/apps/studio.giselles.ai/package.json
index 1dfff5289b..86b47622e0 100644
--- a/apps/studio.giselles.ai/package.json
+++ b/apps/studio.giselles.ai/package.json
@@ -16,9 +16,12 @@
 	},
 	"packageManager": "pnpm@10.0.0",
 	"dependencies": {
+		"@ai-sdk/openai": "catalog:",
 		"@giselle-internal/workflow-designer-ui": "workspace:*",
 		"@giselle-sdk/data-type": "workspace:*",
 		"@giselle-sdk/giselle-engine": "workspace:*",
+		"@giselle-sdk/rag": "workspace:*",
+		"@giselle-sdk/github-tool": "workspace:*",
 		"@giselle-sdk/integration": "workspace:*",
 		"@giselle-sdk/supabase-driver": "workspace:*",
 		"@icons-pack/react-simple-icons": "10.0.0",
diff --git a/apps/studio.giselles.ai/vercel.json b/apps/studio.giselles.ai/vercel.json
new file mode 100644
index 0000000000..c574bc3405
--- /dev/null
+++ b/apps/studio.giselles.ai/vercel.json
@@ -0,0 +1,8 @@
+{
+	"crons": [
+		{
+			"path": "/api/vector-stores/github/ingest",
+			"schedule": "*/10 * * * *"
+		}
+	]
+}
diff --git a/docs/packages-license.md b/docs/packages-license.md
index 8965113920..6122056068 100644
--- a/docs/packages-license.md
+++ b/docs/packages-license.md
@@ -6833,7 +6833,7 @@ CC-BY-4.0 permitted
 
 
 <a name="es-module-lexer"></a>
-### es-module-lexer v1.6.0
+### es-module-lexer v1.7.0
 #### 
 
 ##### Paths
diff --git a/packages/github-tool/package.json b/packages/github-tool/package.json
index 521a936310..b784b88bc4 100644
--- a/packages/github-tool/package.json
+++ b/packages/github-tool/package.json
@@ -29,6 +29,7 @@
 		"tsup": "catalog:"
 	},
 	"dependencies": {
+		"@giselle-sdk/rag": "workspace:*",
 		"@octokit/auth-app": "catalog:",
 		"@octokit/core": "catalog:",
 		"@octokit/graphql": "catalog:",
diff --git a/packages/github-tool/src/blob-loader.ts b/packages/github-tool/src/blob-loader.ts
new file mode 100644
index 0000000000..546518682b
--- /dev/null
+++ b/packages/github-tool/src/blob-loader.ts
@@ -0,0 +1,253 @@
+import type { ContentLoader, LoaderResult } from "@giselle-sdk/rag";
+import type { Octokit } from "@octokit/core";
+
+/**
+ * blob loader metadata
+ */
+export interface GitHubBlobMetadata {
+	owner: string;
+	repo: string;
+	commitSha: string;
+	fileSha: string;
+	path: string;
+	nodeId: string;
+}
+
+/**
+ * Parameters for loading a GitHub blob
+ */
+interface GitHubLoadBlobParams {
+	owner: string;
+	repo: string;
+	path: string;
+	fileSha: string;
+}
+
+/**
+ * Result of a GitHub blob loading operation
+ */
+interface GitHubBlobResult {
+	content: string;
+	metadata: GitHubBlobMetadata;
+}
+
+/**
+ * GitHub repository loading parameters
+ */
+export interface GithubRepositoryParams {
+	owner: string;
+	repo: string;
+	commitSha?: string;
+	baseCommitSha?: string; // for diffing
+}
+
+/**
+ * GitHub repository loader that streams files
+ */
+export class GitHubBlobLoader
+	implements ContentLoader<GithubRepositoryParams, GitHubBlobMetadata>
+{
+	private octokit: Octokit;
+	private options: { maxBlobSize: number };
+
+	constructor(
+		octokit: Octokit,
+		options: {
+			maxBlobSize: number;
+		},
+	) {
+		this.octokit = octokit;
+		this.options = options;
+	}
+
+	/**
+	 * Load content from a repository as a stream
+	 */
+	async *loadStream(
+		params: GithubRepositoryParams,
+	): AsyncIterable<LoaderResult<GitHubBlobMetadata>> {
+		const { owner, repo } = params;
+		let commitSha = params.commitSha;
+		if (!commitSha) {
+			const defaultBranchHead = await fetchDefaultBranchHead(
+				this.octokit,
+				owner,
+				repo,
+			);
+			commitSha = defaultBranchHead.sha;
+		}
+
+		console.log(`Loading repository ${owner}/${repo} at commit ${commitSha}`);
+
+		// Traverse the repository tree
+		for await (const entry of traverseTree(
+			this.octokit,
+			owner,
+			repo,
+			commitSha,
+		)) {
+			const { path, type, sha: fileSha, size } = entry;
+
+			// Process only blob entries (files)
+			if (type !== "blob" || !fileSha || !size || !path) {
+				continue;
+			}
+
+			// Skip files that are too large
+			if (size > this.options.maxBlobSize) {
+				console.warn(
+					`Blob size is too large: ${size} bytes, skipping: ${path}`,
+				);
+				continue;
+			}
+
+			// Load the blob
+			const blob = await loadBlob(
+				this.octokit,
+				{ owner, repo, path, fileSha },
+				commitSha,
+			);
+
+			// Skip binary files
+			if (blob === null) {
+				continue;
+			}
+
+			// Yield as document
+			yield {
+				content: blob.content,
+				metadata: blob.metadata,
+			};
+		}
+	}
+}
+
+/**
+ * Loads a GitHub blob (file) by SHA
+ */
+async function loadBlob(
+	octokit: Octokit,
+	params: GitHubLoadBlobParams,
+	commitSha: string,
+	currentAttempt = 0,
+	maxAttempt = 3,
+): Promise<GitHubBlobResult | null> {
+	const { owner, repo, path, fileSha } = params;
+
+	// Fetch blob from GitHub API
+	// Note: This endpoint supports blobs up to 100 megabytes in size.
+	// https://docs.github.com/en/rest/git/blobs#get-a-blob
+	const { data: blobData, status } = await octokit.request(
+		"GET /repos/{owner}/{repo}/git/blobs/{file_sha}",
+		{
+			owner,
+			repo,
+			file_sha: fileSha,
+		},
+	);
+
+	// Handle server errors with retry logic
+	if (status >= 500) {
+		if (currentAttempt >= maxAttempt) {
+			throw new Error(
+				`Network error: ${status} when fetching ${owner}/${repo}/${fileSha}`,
+			);
+		}
+		// exponential backoff
+		await new Promise((resolve) =>
+			setTimeout(resolve, 2 ** currentAttempt * 100),
+		);
+		return loadBlob(octokit, params, commitSha, currentAttempt + 1, maxAttempt);
+	}
+
+	// Only support base64 encoded content
+	if (blobData.encoding !== "base64") {
+		return null;
+	}
+
+	// Decode base64 content
+	const contentInBytes = Buffer.from(blobData.content, "base64");
+
+	// Check if the content is binary
+	// We use the TextDecoder with fatal option to detect non-text content
+	const textDecoder = new TextDecoder("utf-8", { fatal: true });
+	try {
+		const decodedContent = textDecoder.decode(contentInBytes);
+		return {
+			content: decodedContent,
+			metadata: {
+				owner,
+				repo,
+				commitSha,
+				fileSha,
+				path,
+				nodeId: blobData.node_id,
+			},
+		};
+	} catch (error: unknown) {
+		// Binary content will throw an error when trying to decode
+		return null;
+	}
+}
+
+/**
+ * Iterator for traversing a GitHub repository tree
+ */
+async function* traverseTree(
+	octokit: Octokit,
+	owner: string,
+	repo: string,
+	treeSha: string,
+) {
+	const { data: treeData } = await octokit.request(
+		"GET /repos/{owner}/{repo}/git/trees/{tree_sha}",
+		{
+			owner,
+			repo,
+			tree_sha: treeSha,
+			recursive: "true",
+		},
+	);
+
+	if (treeData.truncated) {
+		/**
+		 * The limit for the tree array is 100,000 entries with a maximum size of 7 MB when using the recursive parameter.
+		 * https://docs.github.com/en/rest/git/trees#get-a-tree
+		 *
+		 * If this limit is exceeded, please consider another way to ingest the repository.
+		 * For example, you can use the git clone or GET tarball API for first time ingestion.
+		 */
+		throw new Error(`Tree is truncated: ${owner}/${repo}/${treeData.sha}`);
+	}
+
+	for (const entry of treeData.tree) {
+		yield entry;
+	}
+}
+
+/**
+ * Get the default branch HEAD commit for a GitHub repository
+ */
+async function fetchDefaultBranchHead(
+	octokit: Octokit,
+	owner: string,
+	repo: string,
+) {
+	const { data: repoData } = await octokit.request(
+		"GET /repos/{owner}/{repo}",
+		{
+			owner,
+			repo,
+		},
+	);
+	const defaultBranch = repoData.default_branch;
+	const { data: branchData } = await octokit.request(
+		"GET /repos/{owner}/{repo}/branches/{branch}",
+		{
+			owner,
+			repo,
+			branch: defaultBranch,
+		},
+	);
+	return branchData.commit;
+}
diff --git a/packages/github-tool/src/index.ts b/packages/github-tool/src/index.ts
index 863bb68842..64255d83c9 100644
--- a/packages/github-tool/src/index.ts
+++ b/packages/github-tool/src/index.ts
@@ -1,9 +1,10 @@
 import { graphql as gql } from "./graphql";
-export * from "./types";
+export * from "./blob-loader";
+export * from "./issues";
 export * from "./octokit";
-export * from "./tools";
 export * from "./repository";
-export * from "./issues";
+export * from "./tools";
+export * from "./types";
 
 export const IssueNodeIdQuery = gql(/* GraphQL */ `
   query IssueNodeIdQuery($name: String!, $owner: String!, $issueNumber: Int!) {
diff --git a/packages/github-tool/turbo.json b/packages/github-tool/turbo.json
index ad1c557a21..10e9f53acd 100644
--- a/packages/github-tool/turbo.json
+++ b/packages/github-tool/turbo.json
@@ -5,7 +5,7 @@
 			"outputs": ["src/graphql-env.d.ts"]
 		},
 		"build": {
-			"dependsOn": ["generate-output"],
+			"dependsOn": ["^build", "generate-output"],
 			"outputs": ["**/dist/**"]
 		}
 	}
diff --git a/packages/rag/package.json b/packages/rag/package.json
new file mode 100644
index 0000000000..bf65cdbe74
--- /dev/null
+++ b/packages/rag/package.json
@@ -0,0 +1,33 @@
+{
+	"name": "@giselle-sdk/rag",
+	"version": "0.0.0",
+	"private": true,
+	"main": "./dist/index.js",
+	"module": "./dist/index.mjs",
+	"types": "./dist/index.d.ts",
+	"scripts": {
+		"build": "tsup",
+		"check-types": "tsc --noEmit",
+		"clean": "rm -rf .turbo dist node_modules",
+		"format": "biome check --write ."
+	},
+	"exports": {
+		"./package.json": "./package.json",
+		".": {
+			"types": "./dist/index.d.ts",
+			"import": "./dist/index.mjs",
+			"require": "./dist/index.js",
+			"default": "./dist/index.js"
+		}
+	},
+	"devDependencies": {
+		"@giselle/giselle-sdk-tsconfig": "workspace:*",
+		"tsup": "catalog:",
+		"vitest": "catalog:",
+		"@types/node": "catalog:"
+	},
+	"dependencies": {
+		"@ai-sdk/openai": "catalog:",
+		"ai": "catalog:"
+	}
+}
diff --git a/packages/rag/src/chunk.ts b/packages/rag/src/chunk.ts
new file mode 100644
index 0000000000..432127d43d
--- /dev/null
+++ b/packages/rag/src/chunk.ts
@@ -0,0 +1,38 @@
+import type { ChunkResult, Chunker } from "./types";
+
+/**
+ * LineChunker splits content into chunks based on line count
+ */
+export class LineChunker implements Chunker {
+	private maxLines: number;
+	private overlap: number;
+
+	constructor(maxLines = 150, overlap = 30) {
+		this.maxLines = maxLines;
+		this.overlap = overlap;
+	}
+
+	/**
+	 * Split document string into chunks by lines
+	 * @param content - document string
+	 */
+	async *chunk(content: string): AsyncGenerator<ChunkResult, void, unknown> {
+		const lines = content.split(/\r?\n/);
+		let chunkIndex = 0;
+
+		for (let i = 0; i < lines.length; i += this.maxLines - this.overlap) {
+			yield {
+				content: lines.slice(i, i + this.maxLines).join("\n"),
+				index: chunkIndex,
+			};
+			chunkIndex++;
+		}
+	}
+}
+
+/**
+ * Helper function to create a LineChunker with default settings
+ */
+export function createLineChunker(maxLines = 150, overlap = 30): Chunker {
+	return new LineChunker(maxLines, overlap);
+}
diff --git a/packages/rag/src/embed.ts b/packages/rag/src/embed.ts
new file mode 100644
index 0000000000..24d4156d67
--- /dev/null
+++ b/packages/rag/src/embed.ts
@@ -0,0 +1,40 @@
+import { openai } from "@ai-sdk/openai";
+import { embed as embedCore } from "ai";
+import type { Embedder } from "./types";
+
+/**
+ * OpenAIEmbedder handles embedding text using OpenAI's embedding models
+ */
+export class OpenAIEmbedder implements Embedder {
+	private model: string;
+	private maxRetries: number;
+
+	constructor(model = "text-embedding-3-small", maxRetries = 2) {
+		this.model = model;
+		this.maxRetries = maxRetries;
+	}
+
+	/**
+	 * Embed a single text using OpenAI's embedding model
+	 * @param text - The text to embed
+	 * @returns The embedded text as an array of numbers
+	 */
+	async embed(text: string): Promise<number[]> {
+		const { embedding } = await embedCore({
+			model: openai.embedding(this.model),
+			value: text,
+			maxRetries: this.maxRetries,
+		});
+		return embedding;
+	}
+}
+
+/**
+ * Helper function to create an OpenAIEmbedder with default settings
+ */
+export function createOpenAIEmbedder(
+	model = "text-embedding-3-small",
+	maxRetries = 2,
+): Embedder {
+	return new OpenAIEmbedder(model, maxRetries);
+}
diff --git a/packages/rag/src/index.ts b/packages/rag/src/index.ts
new file mode 100644
index 0000000000..96e72523f4
--- /dev/null
+++ b/packages/rag/src/index.ts
@@ -0,0 +1,4 @@
+export * from "./chunk";
+export * from "./embed";
+export * from "./ingest";
+export * from "./types";
diff --git a/packages/rag/src/ingest.ts b/packages/rag/src/ingest.ts
new file mode 100644
index 0000000000..22ccbdcd72
--- /dev/null
+++ b/packages/rag/src/ingest.ts
@@ -0,0 +1,132 @@
+import { createLineChunker } from "./chunk";
+import { createOpenAIEmbedder } from "./embed";
+import type {
+	BaseEmbedding,
+	Chunker,
+	ContentLoader,
+	Embedder,
+	EmbeddingStore,
+	EmbeddingTransformer,
+	LoaderResult,
+} from "./types";
+
+/**
+ * Main ingestion function - Loads content from ContentLoader, processes and stores it
+ */
+export async function ingest<
+	T,
+	LoaderMetadataType = Record<string, unknown>,
+	StoreDataType = unknown,
+>(params: {
+	// Data source
+	source: T;
+	// Content loader
+	loader: ContentLoader<T, LoaderMetadataType>;
+	// Embedding store
+	store: EmbeddingStore<StoreDataType>;
+	// Transformation function
+	transformEmbedding?: EmbeddingTransformer<LoaderMetadataType, StoreDataType>;
+}): Promise<void> {
+	const {
+		source,
+		loader,
+		store,
+		transformEmbedding = defaultTransform as EmbeddingTransformer<
+			LoaderMetadataType,
+			StoreDataType
+		>,
+	} = params;
+
+	// Create chunker and embedder
+	const chunker = createLineChunker();
+
+	const embedder = createOpenAIEmbedder();
+
+	// Set content size limit
+	const maxContentSize = 1 * 1024 * 1024; // 1MB
+
+	try {
+		// Start ingestion process
+		await store.startIngestion({ source });
+
+		// Load documents from stream
+		for await (const document of loader.loadStream(source)) {
+			await processDocument(document, {
+				chunker,
+				embedder,
+				store,
+				transformEmbedding,
+				maxContentSize,
+			});
+		}
+
+		// Complete ingestion process
+		await store.completeIngestion({ source });
+	} catch (error) {
+		// Error handling
+		if (error instanceof Error) {
+			await store.failIngestion({ source }, error);
+		} else {
+			await store.failIngestion(
+				{ source },
+				new Error(`Unknown error: ${String(error)}`),
+			);
+		}
+		throw error;
+	}
+}
+
+// Process a single document
+async function processDocument<LoaderMetadataType, StoreDataType>(
+	document: LoaderResult<LoaderMetadataType>,
+	params: {
+		chunker: Chunker;
+		embedder: Embedder;
+		store: EmbeddingStore<StoreDataType>;
+		transformEmbedding: EmbeddingTransformer<LoaderMetadataType, StoreDataType>;
+		maxContentSize: number;
+	},
+): Promise<void> {
+	const { chunker, embedder, store, transformEmbedding, maxContentSize } =
+		params;
+	const { content, metadata } = document;
+
+	// Check content size
+	if (content.length > maxContentSize) {
+		throw new Error(
+			`Content size is too large: ${content.length} bytes, maximum allowed: ${maxContentSize}`,
+		);
+	}
+
+	// Process chunks
+	for await (const chunk of chunker.chunk(content)) {
+		// Skip empty chunks
+		if (chunk.content.length === 0) {
+			continue;
+		}
+
+		// Embed the chunk
+		const embedding = await embedder.embed(chunk.content);
+
+		// Create base embedding object
+		const baseEmbedding: BaseEmbedding = {
+			chunkContent: chunk.content,
+			chunkIndex: chunk.index,
+			embedding,
+		};
+
+		// Transform and store
+		const storeData = transformEmbedding(baseEmbedding, metadata);
+		await store.insertEmbedding(storeData);
+	}
+}
+
+function defaultTransform<T>(
+	baseEmbedding: BaseEmbedding,
+	metadata: Record<string, unknown>,
+): T {
+	return {
+		...baseEmbedding,
+		metadata,
+	} as T;
+}
diff --git a/packages/rag/src/types.ts b/packages/rag/src/types.ts
new file mode 100644
index 0000000000..25a0bc4fea
--- /dev/null
+++ b/packages/rag/src/types.ts
@@ -0,0 +1,67 @@
+/**
+ * Interface for content loader implementations
+ * Responsible for loading content from various sources
+ */
+export interface ContentLoader<T, MetadataType = Record<string, unknown>> {
+	loadStream(params: T): AsyncIterable<LoaderResult<MetadataType>>;
+}
+
+/**
+ * Result of a content loader operation
+ */
+export interface LoaderResult<MetadataType = Record<string, unknown>> {
+	content: string;
+	metadata: MetadataType;
+}
+
+/**
+ * Interface for chunking strategies
+ */
+export interface Chunker {
+	chunk(content: string): AsyncGenerator<ChunkResult, void, unknown>;
+}
+
+/**
+ * Result of a chunking operation
+ */
+export interface ChunkResult {
+	content: string;
+	index: number;
+}
+
+/**
+ * Interface for embedding strategy
+ */
+export interface Embedder {
+	embed(text: string): Promise<number[]>;
+}
+
+/**
+ * Interface for embedding storage
+ */
+export interface EmbeddingStore<T = unknown> {
+	startIngestion(params: Record<string, unknown>): Promise<void>;
+	completeIngestion(params: Record<string, unknown>): Promise<void>;
+	failIngestion(params: Record<string, unknown>, error: Error): Promise<void>;
+	insertEmbedding(data: T): Promise<void>;
+	updateEmbedding(data: T): Promise<void>;
+	deleteEmbedding(key: Partial<T>): Promise<void>;
+}
+
+/**
+ * Base embedding data structure
+ */
+export interface BaseEmbedding {
+	id?: string;
+	chunkContent: string;
+	chunkIndex: number;
+	embedding: number[];
+}
+
+/**
+ * Interface for transforming embeddings
+ */
+export type EmbeddingTransformer<LoaderMetadataType, StoreDataType> = (
+	baseEmbedding: BaseEmbedding,
+	metadata: LoaderMetadataType,
+) => StoreDataType;
diff --git a/packages/rag/tsconfig.json b/packages/rag/tsconfig.json
new file mode 100644
index 0000000000..141aa6680e
--- /dev/null
+++ b/packages/rag/tsconfig.json
@@ -0,0 +1,9 @@
+{
+	"extends": "../../tools/tsconfig/node.json",
+	"compilerOptions": {
+		"outDir": "./dist",
+		"rootDir": "./src"
+	},
+	"include": ["src/**/*.ts"],
+	"exclude": ["node_modules", "dist"]
+}
diff --git a/packages/rag/tsup.config.ts b/packages/rag/tsup.config.ts
new file mode 100644
index 0000000000..bb1af550d2
--- /dev/null
+++ b/packages/rag/tsup.config.ts
@@ -0,0 +1,8 @@
+import { defineConfig } from "tsup";
+
+export default defineConfig({
+	entry: ["src/index.ts"],
+	format: ["cjs", "esm"],
+	dts: true,
+	sourcemap: true,
+});
diff --git a/packages/rag/turbo.json b/packages/rag/turbo.json
new file mode 100644
index 0000000000..d190fa51a0
--- /dev/null
+++ b/packages/rag/turbo.json
@@ -0,0 +1,9 @@
+{
+	"extends": ["//"],
+	"tasks": {
+		"build": {
+			"outputs": ["**/dist/**"]
+		},
+		"test": {}
+	}
+}
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 9407b772ec..1e75a48390 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -339,6 +339,9 @@ importers:
 
   apps/studio.giselles.ai:
     dependencies:
+      '@ai-sdk/openai':
+        specifier: 'catalog:'
+        version: 1.3.18(zod@3.24.1)
       '@giselle-internal/workflow-designer-ui':
         specifier: workspace:*
         version: link:../../internal-packages/workflow-designer-ui
@@ -348,9 +351,15 @@ importers:
       '@giselle-sdk/giselle-engine':
         specifier: workspace:*
         version: link:../../packages/giselle-engine
+      '@giselle-sdk/github-tool':
+        specifier: workspace:*
+        version: link:../../packages/github-tool
       '@giselle-sdk/integration':
         specifier: workspace:*
         version: link:../../packages/integration
+      '@giselle-sdk/rag':
+        specifier: workspace:*
+        version: link:../../packages/rag
       '@giselle-sdk/supabase-driver':
         specifier: workspace:*
         version: link:../../packages/supabase-driver
@@ -964,6 +973,9 @@ importers:
 
   packages/github-tool:
     dependencies:
+      '@giselle-sdk/rag':
+        specifier: workspace:*
+        version: link:../rag
       '@octokit/auth-app':
         specifier: 'catalog:'
         version: 7.1.1
@@ -1055,6 +1067,28 @@ importers:
         specifier: 'catalog:'
         version: 3.0.5(@types/debug@4.1.12)(@types/node@22.15.3)(jiti@2.4.2)(jsdom@26.0.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(lightningcss@1.29.1)(terser@5.39.0)(yaml@2.7.0)
 
+  packages/rag:
+    dependencies:
+      '@ai-sdk/openai':
+        specifier: 'catalog:'
+        version: 1.3.18(zod@3.24.1)
+      ai:
+        specifier: 'catalog:'
+        version: 4.3.9(react@19.1.0)(zod@3.24.1)
+    devDependencies:
+      '@giselle/giselle-sdk-tsconfig':
+        specifier: workspace:*
+        version: link:../../tools/tsconfig
+      '@types/node':
+        specifier: 'catalog:'
+        version: 22.14.0
+      tsup:
+        specifier: 'catalog:'
+        version: 8.3.5(jiti@2.4.2)(postcss@8.5.1)(typescript@5.7.3)(yaml@2.7.0)
+      vitest:
+        specifier: 'catalog:'
+        version: 3.0.5(@types/debug@4.1.12)(@types/node@22.14.0)(jiti@2.4.2)(jsdom@26.0.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(lightningcss@1.29.1)(terser@5.39.0)(yaml@2.7.0)
+
   packages/supabase-driver:
     dependencies:
       '@supabase/supabase-js':
@@ -5967,9 +6001,6 @@ packages:
     resolution: {integrity: sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==}
     engines: {node: '>= 0.4'}
 
-  es-module-lexer@1.6.0:
-    resolution: {integrity: sha512-qqnD1yMU6tk/jnaMosogGySTZP8YtUgAffA9nMN+E/rjxcfRQ6IEk7IiozUjgxKoFHBGjTLnrHB/YC45r/59EQ==}
-
   es-module-lexer@1.7.0:
     resolution: {integrity: sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==}
 
@@ -13201,7 +13232,7 @@ snapshots:
 
   '@types/connect@3.4.36':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/cookie@0.6.0': {}
 
@@ -13253,7 +13284,7 @@ snapshots:
   '@types/glob@7.2.0':
     dependencies:
       '@types/minimatch': 5.1.2
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/hast@3.0.4':
     dependencies:
@@ -13287,11 +13318,11 @@ snapshots:
 
   '@types/mysql@2.15.26':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/node-fetch@2.6.12':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       form-data: 4.0.1
 
   '@types/node@12.20.55': {}
@@ -13310,7 +13341,7 @@ snapshots:
 
   '@types/nodemailer@6.4.17':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/pg-pool@2.0.6':
     dependencies:
@@ -13324,13 +13355,13 @@ snapshots:
 
   '@types/pg@8.11.6':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       pg-protocol: 1.8.0
       pg-types: 4.0.2
 
   '@types/pg@8.6.1':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       pg-protocol: 1.8.0
       pg-types: 2.2.0
 
@@ -13350,11 +13381,11 @@ snapshots:
 
   '@types/tedious@4.0.14':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/through@0.0.33':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@types/tinycolor2@1.4.6': {}
 
@@ -13368,7 +13399,7 @@ snapshots:
 
   '@types/ws@8.5.13':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
 
   '@ungap/structured-clone@1.2.1': {}
 
@@ -14417,8 +14448,6 @@ snapshots:
 
   es-errors@1.3.0: {}
 
-  es-module-lexer@1.6.0: {}
-
   es-module-lexer@1.7.0: {}
 
   es-object-atoms@1.1.1:
@@ -14556,7 +14585,7 @@ snapshots:
 
   estree-walker@3.0.3:
     dependencies:
-      '@types/estree': 1.0.6
+      '@types/estree': 1.0.7
 
   esutils@2.0.3: {}
 
@@ -16811,7 +16840,7 @@ snapshots:
       '@protobufjs/path': 1.1.2
       '@protobufjs/pool': 1.1.0
       '@protobufjs/utf8': 1.1.0
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       long: 5.2.4
 
   proxy-agent@6.5.0:
@@ -17472,7 +17501,7 @@ snapshots:
 
   stripe@17.4.0:
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       qs: 6.14.0
 
   style-to-object@1.0.8:
@@ -18001,7 +18030,7 @@ snapshots:
     dependencies:
       cac: 6.7.14
       debug: 4.4.0
-      es-module-lexer: 1.6.0
+      es-module-lexer: 1.7.0
       pathe: 2.0.2
       vite: 6.0.11(@types/node@22.14.0)(jiti@2.4.2)(lightningcss@1.29.1)(terser@5.39.0)(yaml@2.7.0)
     transitivePeerDependencies:
@@ -18022,7 +18051,7 @@ snapshots:
     dependencies:
       cac: 6.7.14
       debug: 4.4.0
-      es-module-lexer: 1.6.0
+      es-module-lexer: 1.7.0
       pathe: 2.0.2
       vite: 6.0.11(@types/node@22.15.3)(jiti@2.4.2)(lightningcss@1.29.1)(terser@5.39.0)(yaml@2.7.0)
     transitivePeerDependencies:


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR adds GitHub Vector Store functionality (blobs only) by implementing ingestion, embedding, and chunking logic along with a management UI for repository registration and deletion. Key changes include new modules for handling GitHub code ingestion, API endpoint integration with Vercel Cron, and updates to the UI and routing for managing the vector store.

### Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

| File | Description |
| ---- | ----------- |
| packages/github-vector-store | New modules for embedding, chunking, and ingestion of GitHub code |
| apps/studio.giselles.ai | New API endpoint, UI pages, actions, and navigation updates to support vector store management |
| docs/packages-license.md | Package version update for licensing documentation |




state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T06:14:11Z
user: login: satococoa
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T06:14:48Z
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR introduces the initial implementation for a GitHub Vector Store (Blobs only) feature. Key changes include adding the API endpoint for periodic GitHub code ingestion, implementing the vector store functionality in a dedicated package, and integrating the feature into the Studio management UI with a feature flag.

### Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

| File | Description |
| ---- | ----------- |
| packages/github-vector-store/** | New module exposing vector store functionality (chunking, embedding, blob ingestion) |
| apps/studio.giselles.ai/** | Integration of the vector store feature into the Studio management screens and API endpoints |
| docs/packages-license.md | Updated package license version for es-module-lexer |
| Other configuration and flag files | Updated to conditionally enable the new vector store functionality |




state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T07:48:24Z
user: login: toyamarinyon
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T07:48:25Z
user: login: satococoa
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T08:01:17Z
user: login: toyamarinyon
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T08:03:50Z
user: login: satococoa
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T08:06:19Z
user: login: shige
body: Thank you! 🆙 

I was able to confirm that it works with the current code. ✅ 

<img width="953" alt="image" src="https://github.com/user-attachments/assets/7cc5797a-df5f-44f7-bb2a-26ec5774fea8" />

state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-14T01:20:14Z
user: login: toyamarinyon
body: The current RAG implementation in `ingest.ts` and GitHub repository integration has several significant issues:

1. **Poorly Defined Types**
   - Generic type `T` has no constraints
   - Overuse of `Record<string, unknown>` and `unknown` types
   - Unsafe type assertions with `as`

2. **Unclear Responsibilities**
   - Single file contains API handler, data transformation, and storage logic
   - `source: T` parameter is passed around without clear purpose

3. **Complex Error Handling**
   - Error handling logic is scattered and inconsistent
   - DB retry logic embedded in class implementation

4. **Monolithic Design**
   - Tightly coupled components make testing and extension difficult
   - Poor separation of concerns

## Proposed Solution: Mastra-style Implementation

I recommend refactoring to a cleaner, more modular approach similar to Mastra's RAG implementation, which has a clear step-by-step flow:

```typescript
// Mastra example
import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";

// 1. Initialize document
const doc = MDocument.fromText(`Your document text here...`);

// 2. Create chunks
const chunks = await doc.chunk({
  strategy: "recursive",
  size: 512,
  overlap: 50,
});

// 3. Generate embeddings
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});

// 4. Store in vector database
const pgVector = new PgVector(process.env.POSTGRES_CONNECTION_STRING);
await pgVector.upsert({
  indexName: "embeddings",
  vectors: embeddings,
});
```

## Implementation Recommendations

1. **Create Modular Document Handling**
   - `GiselleDocument` class with static factory methods
   - Define clear metadata schema with Zod
   - Support various chunking strategies

2. **Simplify Vector Store Implementation**
   - Create dedicated `PgVectorStore` class
   - Implement clear upsert and query methods
   - Proper error handling and transactions

3. **Improved GitHub Integration**
   - `GitHubRepository` class for API interaction
   - `GitHubDocuments` class for document conversion
   - Clear filtering options for repositories

4. **Direct Dependency on OpenAI SDK**
   - Import directly from `@ai-sdk/openai` rather than wrapping

5. **API Endpoint Simplification**
   - Clear separation of authentication, data fetching, and processing
   - Better error handling and status management

## Benefits

1. **Type Safety**: Properly defined interfaces and Zod schemas
2. **Modularity**: Independent, testable components
3. **Clarity**: Intuitive step-by-step flow
4. **Flexibility**: Easy to customize or replace individual components
5. **Maintainability**: Simpler code structure that's easier to understand and extend

## Implementation Examples

### 1. Document Processing

```typescript
// @giselle-sdk/rag/src/document.ts
import { z } from "zod";

export const DocMetadataSchema = z.object({
  id: z.string().optional(),
  source: z.string().optional(),
  title: z.string().optional(),
  url: z.string().url().optional(),
  created: z.date().optional(),
  author: z.string().optional(),
  [z.string()]: z.unknown(),
});

export type DocMetadata = z.infer<typeof DocMetadataSchema>;

export type ChunkOptions = {
  strategy: "recursive" | "fixed" | "line";
  size: number;
  overlap?: number;
};

export interface DocumentChunk {
  id: string;
  text: string;
  index: number;
  metadata: DocMetadata;
}

export class GiselleDocument {
  text: string;
  metadata: DocMetadata;

  constructor(text: string, metadata: DocMetadata = {}) {
    this.text = text;
    this.metadata = metadata;
  }

  static fromText(text: string, metadata: DocMetadata = {}): GiselleDocument {
    return new GiselleDocument(text, metadata);
  }
  
  static fromHTML(html: string, metadata: DocMetadata = {}): GiselleDocument {
    const text = convertHTMLToText(html);
    return new GiselleDocument(text, metadata);
  }

  async chunk(options: ChunkOptions): Promise<DocumentChunk[]> {
    const { strategy, size, overlap = 0 } = options;
    
    switch (strategy) {
      case "recursive":
        return recursiveChunking(this.text, size, overlap, this.metadata);
      case "fixed":
        return fixedChunking(this.text, size, overlap, this.metadata);
      case "line":
        return lineChunking(this.text, size, overlap, this.metadata);
      default:
        throw new Error(`Unknown chunking strategy: ${strategy}`);
    }
  }
}
```

### 2. Vector Store Implementation

```typescript
// @giselle-sdk/rag/src/stores/pg-vector.ts
import { Pool } from "pg";
import { v4 as uuidv4 } from "uuid";

export interface VectorStoreOptions {
  connectionString: string;
  schema?: string;
}

export interface UpsertOptions {
  indexName: string;
  vectors: number[][];
  metadata?: Record<string, any>[];
  ids?: string[];
}

export interface QueryOptions {
  indexName: string;
  queryVector: number[];
  topK: number;
  filter?: Record<string, any>;
}

export class PgVectorStore {
  private pool: Pool;
  private schema: string;

  constructor(options: VectorStoreOptions) {
    this.pool = new Pool({
      connectionString: options.connectionString,
    });
    this.schema = options.schema || "public";
  }

  async upsert(options: UpsertOptions): Promise<void> {
    const { indexName, vectors, metadata = [], ids = [] } = options;
    
    const finalMetadata = metadata.length === vectors.length 
      ? metadata 
      : vectors.map(() => ({}));
    
    const finalIds = ids.length === vectors.length 
      ? ids 
      : vectors.map(() => uuidv4());
    
    const client = await this.pool.connect();
    try {
      await client.query("BEGIN");
      
      for (let i = 0; i < vectors.length; i++) {
        await client.query(
          `INSERT INTO ${this.schema}.${indexName} (id, vector, metadata)
           VALUES ($1, $2, $3)
           ON CONFLICT (id) DO UPDATE
           SET vector = $2, metadata = $3`,
          [finalIds[i], JSON.stringify(vectors[i]), JSON.stringify(finalMetadata[i])]
        );
      }
      
      await client.query("COMMIT");
    } catch (error) {
      await client.query("ROLLBACK");
      throw error;
    } finally {
      client.release();
    }
  }

  async query(options: QueryOptions): Promise<Array<{ id: string; similarity: number; metadata: any }>> {
    const { indexName, queryVector, topK, filter } = options;
    
    let query = `
      SELECT id, metadata, 1 - (vector <-> $1) as similarity
      FROM ${this.schema}.${indexName}
    `;
    
    if (filter) {
      const conditions = Object.entries(filter)
        .map(([key, value], index) => `metadata->>'${key}' = $${index + 2}`)
        .join(" AND ");
      
      query += ` WHERE ${conditions}`;
    }
    
    query += ` ORDER BY similarity DESC LIMIT $${filter ? Object.keys(filter).length + 2 : 2}`;
    
    const params = [JSON.stringify(queryVector), topK];
    if (filter) {
      params.push(...Object.values(filter));
    }
    
    const { rows } = await this.pool.query(query, params);
    return rows;
  }
}
```

### 3. GitHub Integration

```typescript
// @giselle-sdk/github-tool/src/github-repository.ts
import { Octokit } from "@octokit/rest";
import { createAppAuth } from "@octokit/auth-app";
import { z } from "zod";

export const GitHubRepoConfigSchema = z.object({
  owner: z.string(),
  repo: z.string(),
  ref: z.string().optional(),
  path: z.string().optional(),
});

export type GitHubRepoConfig = z.infer<typeof GitHubRepoConfigSchema>;

export interface GitHubAuthOptions {
  appId?: string;
  privateKey?: string;
  installationId?: number;
  token?: string;
}

export class GitHubRepository {
  private octokit: Octokit;
  private config: GitHubRepoConfig;
  
  constructor(config: GitHubRepoConfig, authOptions: GitHubAuthOptions) {
    this.config = GitHubRepoConfigSchema.parse(config);
    
    if (authOptions.appId && authOptions.privateKey && authOptions.installationId) {
      this.octokit = new Octokit({
        authStrategy: createAppAuth,
        auth: {
          appId: authOptions.appId,
          privateKey: authOptions.privateKey,
          installationId: authOptions.installationId,
        },
      });
    } else if (authOptions.token) {
      this.octokit = new Octokit({
        auth: authOptions.token,
      });
    } else {
      throw new Error("Invalid GitHub authentication options");
    }
  }
  
  async listFiles(): Promise<Array<{ path: string; type: string; sha: string }>> {
    const { owner, repo, path = "", ref } = this.config;
    
    const result = await this.octokit.git.getTree({
      owner,
      repo,
      tree_sha: ref || "HEAD",
      recursive: "1"
    });
    
    if (!result.data.tree) {
      return [];
    }
    
    return result.data.tree
      .filter(item => path === "" || item.path?.startsWith(path))
      .map(item => ({
        path: item.path || "",
        type: item.type || "blob",
        sha: item.sha || ""
      }));
  }
  
  async getFileContent(path: string): Promise<{ content: string; sha: string }> {
    const { owner, repo, ref } = this.config;
    
    const result = await this.octokit.repos.getContent({
      owner,
      repo,
      path,
      ref: ref || undefined
    });
    
    if (Array.isArray(result.data) || !('content' in result.data)) {
      throw new Error(`${path} is not a file`);
    }
    
    const content = Buffer.from(result.data.content, 'base64').toString('utf-8');
    return {
      content,
      sha: result.data.sha
    };
  }
  
  async getCodeFiles(extensions: string[] = []): Promise<Array<{ path: string; sha: string }>> {
    const files = await this.listFiles();
    
    return files
      .filter(file => {
        if (file.type !== "blob") return false;
        if (extensions.length === 0) return true;
        return extensions.some(ext => file.path.endsWith(ext));
      })
      .map(({ path, sha }) => ({ path, sha }));
  }
}

// @giselle-sdk/github-tool/src/github-document.ts
import { GiselleDocument, type DocMetadata } from "@giselle-sdk/rag";
import { GitHubRepository } from "./github-repository";

export interface GitHubDocumentOptions {
  includePaths?: string[];
  excludePaths?: string[];
  extensions?: string[];
  maxFileSizeBytes?: number;
}

export class GitHubDocuments {
  private repo: GitHubRepository;
  private options: GitHubDocumentOptions;
  
  constructor(repo: GitHubRepository, options: GitHubDocumentOptions = {}) {
    this.repo = repo;
    this.options = {
      maxFileSizeBytes: 1024 * 1024, // Default 1MB
      ...options
    };
  }
  
  async getDocuments(): Promise<GiselleDocument[]> {
    const { extensions = [] } = this.options;
    
    const files = await this.repo.getCodeFiles(extensions);
    const filteredFiles = this.filterFilesByPath(files);
    const documents: GiselleDocument[] = [];
    
    for (const file of filteredFiles) {
      try {
        const { content, sha } = await this.repo.getFileContent(file.path);
        
        if (content.length > (this.options.maxFileSizeBytes || Infinity)) {
          console.warn(`Skipping file ${file.path} as it exceeds the size limit`);
          continue;
        }
        
        const metadata: DocMetadata = {
          source: "github",
          filePath: file.path,
          sha: sha,
          fileType: this.getFileType(file.path),
          gitHubRepo: true
        };
        
        const document = GiselleDocument.fromText(content, metadata);
        documents.push(document);
      } catch (error) {
        console.error(`Error processing file ${file.path}:`, error);
      }
    }
    
    return documents;
  }
  
  // Helper methods omitted for brevity...
}
```

### 4. API Endpoint Implementation

```typescript
// apps/studio.giselles.ai/app/api/vector-stores/github/ingest/route.ts
import { GitHubRepository, GitHubDocuments } from "@giselle-sdk/github-tool";
import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { PgVectorStore } from "@giselle-sdk/rag";
import { db, githubRepositoryIndex } from "@/drizzle";
import { eq } from "drizzle-orm";
import type { NextRequest } from "next/server";

export const maxDuration = 800;

export async function GET(request: NextRequest) {
  // Authentication
  const authHeader = request.headers.get("authorization");
  if (authHeader !== `Bearer ${process.env.CRON_SECRET}`) {
    return new Response("Unauthorized", { status: 401 });
  }

  // Fetch repositories to process
  const repos = await db
    .select({
      owner: githubRepositoryIndex.owner,
      repo: githubRepositoryIndex.repo,
      installationId: githubRepositoryIndex.installationId,
      teamDbId: githubRepositoryIndex.teamDbId,
    })
    .from(githubRepositoryIndex)
    .where(eq(githubRepositoryIndex.status, "idle"));

  // Process each repository
  for (const repoInfo of repos) {
    try {
      // Update status to running
      await updateRepoStatus(repoInfo.owner, repoInfo.repo, "running");

      // Setup GitHub repository
      const repo = new GitHubRepository(
        {
          owner: repoInfo.owner,
          repo: repoInfo.repo,
        },
        {
          appId: process.env.GITHUB_APP_ID!,
          privateKey: process.env.GITHUB_APP_PRIVATE_KEY!,
          installationId: repoInfo.installationId,
        }
      );

      // Get documents
      const githubDocs = new GitHubDocuments(repo, {
        excludePaths: ['node_modules/**', '.git/**'],
        maxFileSizeBytes: 1024 * 1024, // 1MB
      });

      const documents = await githubDocs.getDocuments();
      
      // Setup vector store
      const vectorStore = new PgVectorStore({
        connectionString: process.env.DATABASE_URL!,
      });

      // Process all chunks
      let allChunks = [];
      for (const doc of documents) {
        const chunks = await doc.chunk({
          strategy: "recursive",
          size: 512,
          overlap: 50,
        });
        allChunks.push(...chunks);
      }

      // Generate embeddings
      const { embeddings } = await embedMany({
        values: allChunks.map(chunk => chunk.text),
        model: openai.embedding("text-embedding-3-small"),
      });

      // Vector store index name
      const indexName = `github_repo_${repoInfo.teamDbId}`;

      // Store in vector database
      await vectorStore.upsert({
        indexName,
        vectors: embeddings,
        metadata: allChunks.map(chunk => ({
          owner: repoInfo.owner,
          repo: repoInfo.repo,
          path: chunk.metadata.filePath,
          teamDbId: repoInfo.teamDbId,
          chunkIndex: chunk.index,
        })),
      });

      // Update status to completed
      await updateRepoStatus(repoInfo.owner, repoInfo.repo, "completed");
    } catch (error) {
      console.error(`Error processing ${repoInfo.owner}/${repoInfo.repo}:`, error);
      await updateRepoStatus(repoInfo.owner, repoInfo.repo, "failed");
    }
  }

  return new Response("ok", { status: 200 });
}

// Update repository status
async function updateRepoStatus(owner: string, repo: string, status: "idle" | "running" | "completed" | "failed") {
  await db
    .update(githubRepositoryIndex)
    .set({ status })
    .where(
      eq(githubRepositoryIndex.owner, owner) && 
      eq(githubRepositoryIndex.repo, repo)
    );
}
```

## Usage Example

This approach results in a much simpler and cleaner integration:

```typescript
// Example usage
import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { GitHubRepository, GitHubDocuments } from "@giselle-sdk/github-tool";
import { PgVectorStore } from "@giselle-sdk/rag";

async function ingestGitHubRepo() {
  // 1. Setup GitHub repository
  const repo = new GitHubRepository(
    { owner: "organization", repo: "repository" },
    { token: process.env.GITHUB_TOKEN }
  );
  
  // 2. Get documents
  const documents = await new GitHubDocuments(repo).getDocuments();
  
  // 3. Process all documents
  let allChunks = [];
  for (const doc of documents) {
    const chunks = await doc.chunk({
      strategy: "recursive",
      size: 512,
      overlap: 50,
    });
    allChunks.push(...chunks);
  }
  
  // 4. Generate embeddings
  const { embeddings } = await embedMany({
    values: allChunks.map(chunk => chunk.text),
    model: openai.embedding("text-embedding-3-small"),
  });
  
  // 5. Store in vector database
  const vectorStore = new PgVectorStore({
    connectionString: process.env.DATABASE_URL
  });
  
  await vectorStore.upsert({
    indexName: "github_embeddings",
    vectors: embeddings,
    metadata: allChunks.map(chunk => chunk.metadata),
  });
}
```

I believe this approach will significantly improve code quality, maintainability, and developer experience.


state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T08:49:35Z
user: login: toyamarinyon
body: First, I believe it's necessary to keep creating the best possible product at each moment. I've also made breaking changes many times, but I'm always creating what I consider the best solution possible at that time. If there's a way to make something better, I don't postpone it.

However, it's fine to proceed with the current state to expedite verification, but neither Vercel nor GitHub have a feature that forces refactoring before removing a Feature Flag, so I'm wondering how to express that point.

Alternatively, if you prefix all functions with `experimental_` to explicitly indicate that this implementation is experimental, I think the current implementation would be acceptable.


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T09:46:56Z
user: login: toyamarinyon
body: > I approved them, but I couldn't tell the difference.

If so, let's continue as is.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T10:00:31Z

Review comments:
diff_hunk: @@ -0,0 +1,72 @@
+"use server";
+
+import { db, githubRepositoryIndex } from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { buildAppInstallationClient } from "@/services/external/github";
+import { fetchCurrentTeam } from "@/services/teams";
+import { createId } from "@paralleldrive/cuid2";
+import { and, eq } from "drizzle-orm";
+import { revalidatePath } from "next/cache";
+
+export async function registerRepositoryIndex(
+	owner: string,
+	repo: string,
+	installationId: number,
+) {
+	const team = await fetchCurrentTeam();
+
+	// check if the user have access to the installation
+	// FIXME: When the installation is managed by the team, we should use the team's installation instead
+	const githubIdentityState = await getGitHubIdentityState();
+	if (githubIdentityState.status !== "authorized") {
+		throw new Error("User is not authorized to access the repository");
+	}
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	const installation = installationData.installations.find(
+		(installation) => installation.id === installationId,
+	);
+	if (!installation) {
+		throw new Error("Installation not found");
+	}
+
+	// check if the installation can access the repository
+	const installationClient = await buildAppInstallationClient(installationId);
+
+	// check if the repository is already registered
+	const repository = await installationClient.request(
+		"GET /repos/{owner}/{repo}",
+		{
+			owner,
+			repo,
+		},
+	);
+	if (repository.status !== 200) {
+		throw new Error("Repository not found");
+	}
+
+	const newIndexId = `gthbi_${createId()}` as GitHubRepositoryIndexId;
+	await db.insert(githubRepositoryIndex).values({
+		id: newIndexId,
+		owner,
+		repo,
+		teamDbId: team.dbId,
+		installationId,
+	});
+
+	revalidatePath("/settings/team/vector-store");
path: apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
user: login: Copilot
body: The revalidation path '/settings/team/vector-store' appears to be inconsistent with the rest of the vector store pages which use '/settings/team/vector-stores'. Consider updating the path to '/settings/team/vector-stores' to ensure consistent routing.
```suggestion
	revalidatePath("/settings/team/vector-stores");
```
created_at: 2025-05-13T06:14:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,72 @@
+"use server";
+
+import { db, githubRepositoryIndex } from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { buildAppInstallationClient } from "@/services/external/github";
+import { fetchCurrentTeam } from "@/services/teams";
+import { createId } from "@paralleldrive/cuid2";
+import { and, eq } from "drizzle-orm";
+import { revalidatePath } from "next/cache";
+
+export async function registerRepositoryIndex(
+	owner: string,
+	repo: string,
+	installationId: number,
+) {
+	const team = await fetchCurrentTeam();
+
+	// check if the user have access to the installation
+	// FIXME: When the installation is managed by the team, we should use the team's installation instead
+	const githubIdentityState = await getGitHubIdentityState();
+	if (githubIdentityState.status !== "authorized") {
+		throw new Error("User is not authorized to access the repository");
+	}
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	const installation = installationData.installations.find(
+		(installation) => installation.id === installationId,
+	);
+	if (!installation) {
+		throw new Error("Installation not found");
+	}
+
+	// check if the installation can access the repository
+	const installationClient = await buildAppInstallationClient(installationId);
+
+	// check if the repository is already registered
+	const repository = await installationClient.request(
+		"GET /repos/{owner}/{repo}",
+		{
+			owner,
+			repo,
+		},
+	);
+	if (repository.status !== 200) {
+		throw new Error("Repository not found");
+	}
+
+	const newIndexId = `gthbi_${createId()}` as GitHubRepositoryIndexId;
+	await db.insert(githubRepositoryIndex).values({
+		id: newIndexId,
+		owner,
+		repo,
+		teamDbId: team.dbId,
+		installationId,
+	});
+
+	revalidatePath("/settings/team/vector-store");
+}
+
+export async function deleteRepositoryIndex(indexId: GitHubRepositoryIndexId) {
+	const team = await fetchCurrentTeam();
+	await db
+		.delete(githubRepositoryIndex)
+		.where(
+			and(
+				eq(githubRepositoryIndex.teamDbId, team.dbId),
+				eq(githubRepositoryIndex.id, indexId),
+			),
+		);
+	revalidatePath("/settings/team/vector-store");
path: apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
user: login: Copilot
body: The revalidation path '/settings/team/vector-store' should be updated to '/settings/team/vector-stores' to match the routing used throughout the application.
```suggestion
	revalidatePath("/settings/team/vector-stores");
```
created_at: 2025-05-13T06:14:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,72 @@
+"use server";
+
+import { db, githubRepositoryIndex } from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { buildAppInstallationClient } from "@/services/external/github";
+import { fetchCurrentTeam } from "@/services/teams";
+import { createId } from "@paralleldrive/cuid2";
+import { and, eq } from "drizzle-orm";
+import { revalidatePath } from "next/cache";
+
+export async function registerRepositoryIndex(
+	owner: string,
+	repo: string,
+	installationId: number,
+) {
+	const team = await fetchCurrentTeam();
+
+	// check if the user have access to the installation
+	// FIXME: When the installation is managed by the team, we should use the team's installation instead
+	const githubIdentityState = await getGitHubIdentityState();
+	if (githubIdentityState.status !== "authorized") {
+		throw new Error("User is not authorized to access the repository");
+	}
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	const installation = installationData.installations.find(
+		(installation) => installation.id === installationId,
+	);
+	if (!installation) {
+		throw new Error("Installation not found");
+	}
+
+	// check if the installation can access the repository
+	const installationClient = await buildAppInstallationClient(installationId);
+
+	// check if the repository is already registered
+	const repository = await installationClient.request(
+		"GET /repos/{owner}/{repo}",
+		{
+			owner,
+			repo,
+		},
+	);
+	if (repository.status !== 200) {
+		throw new Error("Repository not found");
+	}
+
+	const newIndexId = `gthbi_${createId()}` as GitHubRepositoryIndexId;
+	await db.insert(githubRepositoryIndex).values({
+		id: newIndexId,
+		owner,
+		repo,
+		teamDbId: team.dbId,
+		installationId,
+	});
+
+	revalidatePath("/settings/team/vector-store");
path: apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
user: login: satococoa
body: nice catch 🧤 
created_at: 2025-05-13T06:14:48Z
author_association: CONTRIBUTOR
path: packages/github-vector-store/package.json
user: login: toyamarinyon
body: I think it's not good that the implementation of ingest and connector are bundled together, and that it uses GitHub-specific naming like GitHubVectorStore.
created_at: 2025-05-13T07:48:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,72 @@
+"use server";
+
+import { db, githubRepositoryIndex } from "@/drizzle";
+import type { GitHubRepositoryIndexId } from "@/packages/types";
+import { getGitHubIdentityState } from "@/services/accounts";
+import { buildAppInstallationClient } from "@/services/external/github";
+import { fetchCurrentTeam } from "@/services/teams";
+import { createId } from "@paralleldrive/cuid2";
+import { and, eq } from "drizzle-orm";
+import { revalidatePath } from "next/cache";
+
+export async function registerRepositoryIndex(
+	owner: string,
+	repo: string,
+	installationId: number,
+) {
+	const team = await fetchCurrentTeam();
+
+	// check if the user have access to the installation
+	// FIXME: When the installation is managed by the team, we should use the team's installation instead
+	const githubIdentityState = await getGitHubIdentityState();
+	if (githubIdentityState.status !== "authorized") {
+		throw new Error("User is not authorized to access the repository");
+	}
+	const userClient = githubIdentityState.gitHubUserClient;
+	const installationData = await userClient.getInstallations();
+	const installation = installationData.installations.find(
+		(installation) => installation.id === installationId,
+	);
+	if (!installation) {
+		throw new Error("Installation not found");
+	}
path: apps/studio.giselles.ai/app/(main)/settings/team/vector-stores/actions.ts
user: login: Copilot
body: Consider addressing the FIXME by planning how team-managed installations will be supported in the future to avoid potential inconsistencies. This will help ensure the code remains robust as the feature evolves.
```suggestion
	// Check if the installation is managed by the team
	const teamInstallation = await fetchTeamGitHubInstallation(team.dbId);
	let installationClient;

	if (teamInstallation && teamInstallation.id === installationId) {
		// Use the team's installation client
		installationClient = await buildAppInstallationClient(installationId);
	} else {
		// Fallback to user-specific installation
		const githubIdentityState = await getGitHubIdentityState();
		if (githubIdentityState.status !== "authorized") {
			throw new Error("User is not authorized to access the repository");
		}
		const userClient = githubIdentityState.gitHubUserClient;
		const installationData = await userClient.getInstallations();
		const installation = installationData.installations.find(
			(installation) => installation.id === installationId,
		);
		if (!installation) {
			throw new Error("Installation not found");
		}
		installationClient = await buildAppInstallationClient(installationId);
	}
```
created_at: 2025-05-13T07:48:24Z
author_association: CONTRIBUTOR
path: packages/github-vector-store/package.json
user: login: satococoa
body: Alright—​to introduce a cleaner abstraction, I’ll break this out into two packages:
- `github-loader`
- `vector-store` (handles ingest, chunking, embedding, and querying)

Does that sound reasonable?
created_at: 2025-05-13T08:01:17Z
author_association: CONTRIBUTOR
path: packages/github-vector-store/package.json
user: login: toyamarinyon
body: I think it would be better if ingest, chunking, embedding, and querying are placed in 'rag' rather than 'vector-store' since these are RAG (Retrieval-Augmented Generation) functions, not vector-store functions. The github-loader could be included in the github-tool.
created_at: 2025-05-13T08:03:50Z
author_association: CONTRIBUTOR
path: packages/github-vector-store/package.json
user: login: satococoa
body: thank you!
created_at: 2025-05-13T08:06:18Z
author_association: CONTRIBUTOR

PR comments:
user: login: changeset-bot[bot]
created_at: 2025-05-08T08:16:41Z
body: ###  ⚠️  No Changeset found

Latest commit: 527976267d254998d506b3d9ce8a56a8c7b3c8ad

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. **If these changes should result in a version bump, you need to add a changeset.**



[Click here to learn what changesets are, and how to add one](https://github.com/changesets/changesets/blob/main/docs/adding-a-changeset.md).

[Click here if you're a maintainer who wants to add a changeset to this PR](https://github.com/giselles-ai/giselle/new/fetch-github-repo-code-worker?filename=.changeset/sharp-toes-sparkle.md&value=---%0A%22%40fake-scope%2Ffake-pkg%22%3A%20patch%0A---%0A%0AAdd%20GitHub%20Vector%20Store%20(Blobs%20only)%0A)

<details><summary>💥 An error occurred when fetching the changed packages and changesets in this PR</summary>

```
Some errors occurred when validating the changesets config:
The package "playground" depends on the ignored package "giselle-sdk", but "playground" is not being ignored. Please add "playground" to the `ignore` option.
The package "studio.giselles.ai" depends on the ignored package "giselle-sdk", but "studio.giselles.ai" is not being ignored. Please add "studio.giselles.ai" to the `ignore` option.
The package "@giselle-internal/workflow-designer-ui" depends on the ignored package "giselle-sdk", but "@giselle-internal/workflow-designer-ui" is not being ignored. Please add "@giselle-internal/workflow-designer-ui" to the `ignore` option.
```

</details>

author_association: NONE
user: login: vercel[bot]
created_at: 2025-05-08T08:16:42Z
body: [vc]: #91Bz5eVQN21eLf1t4C7xE3bau5OkJfiFd5+aVp6Mmd0=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJnaXNlbGxlIiwicm9vdERpcmVjdG9yeSI6ImFwcHMvc3R1ZGlvLmdpc2VsbGVzLmFpIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3IwNi1lZGdlL2dpc2VsbGUvM2FqRm5QTFA4dUx2ckx1VW1pQ2tTUEhKQkdyRCIsInByZXZpZXdVcmwiOiJnaXNlbGxlLWdpdC1mZXRjaC1naXRodWItcmVwby1jb2RlLXdvcmtlci1yMDYtZWRnZS52ZXJjZWwuYXBwIiwibmV4dENvbW1pdFN0YXR1cyI6IkRFUExPWUVEIiwibGl2ZUZlZWRiYWNrIjp7InJlc29sdmVkIjowLCJ1bnJlc29sdmVkIjowLCJ0b3RhbCI6MCwibGluayI6Imdpc2VsbGUtZ2l0LWZldGNoLWdpdGh1Yi1yZXBvLWNvZGUtd29ya2VyLXIwNi1lZGdlLnZlcmNlbC5hcHAifX0seyJuYW1lIjoiZ2lzZWxsZS1wbGF5Z3JvdW5kIiwicm9vdERpcmVjdG9yeSI6ImFwcHMvcGxheWdyb3VuZCIsImluc3BlY3RvclVybCI6Imh0dHBzOi8vdmVyY2VsLmNvbS9yMDYtZWRnZS9naXNlbGxlLXBsYXlncm91bmQvSGlKTlF0S2g3VG9XRmVkTXR6MWIzS2Z3Z0xjUiIsInByZXZpZXdVcmwiOiJnaXNlbGxlLXBsYXlncm91bmQtZ2l0LWZldGNoLWdpdGh1Yi1yZXBvLWNvZGUtd29ya2VyLXIwNi1lZGdlLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiREVQTE9ZRUQiLCJsaXZlRmVlZGJhY2siOnsicmVzb2x2ZWQiOjAsInVucmVzb2x2ZWQiOjAsInRvdGFsIjowLCJsaW5rIjoiZ2lzZWxsZS1wbGF5Z3JvdW5kLWdpdC1mZXRjaC1naXRodWItcmVwby1jb2RlLXdvcmtlci1yMDYtZWRnZS52ZXJjZWwuYXBwIn19XX0=
**The latest updates on your projects**. Learn more about [Vercel for Git ↗︎](https://vercel.link/github-learn-more)

| Name | Status | Preview | Comments | Updated (UTC) |
| :--- | :----- | :------ | :------- | :------ |
| **giselle** | ✅ Ready ([Inspect](https://vercel.com/r06-edge/giselle/3ajFnPLP8uLvrLuUmiCkSPHJBGrD)) | [Visit Preview](https://giselle-git-fetch-github-repo-code-worker-r06-edge.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/giselle-git-fetch-github-repo-code-worker-r06-edge.vercel.app?via=pr-comment-feedback-link) | May 14, 2025 7:59am |
| **giselle-playground** | ✅ Ready ([Inspect](https://vercel.com/r06-edge/giselle-playground/HiJNQtKh7ToWFedMtz1b3KfwgLcR)) | [Visit Preview](https://giselle-playground-git-fetch-github-repo-code-worker-r06-edge.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/giselle-playground-git-fetch-github-repo-code-worker-r06-edge.vercel.app?via=pr-comment-feedback-link) | May 14, 2025 7:59am |



author_association: NONE
user: login: shige
created_at: 2025-05-14T09:36:13Z
body: @toyamarinyon cc/ @satococoa 

This PR is a flagged feature, but do the issues pointed out in the review need to be resolved in this PR? Could you rearchitect it in a separate issue?

I want to try out whether I can realize the use case I have in mind based on this feature, so if it can be released with a flag, I would like to make it available for testing first.
author_association: MEMBER
user: login: shige
created_at: 2025-05-14T09:55:58Z
body: @toyamarinyon 

In that sense, I thought the same thing could be said about other PRs that kept adding half-baked features with feature flags that didn't work yet.

I approved them, but I couldn't tell the difference.
author_association: MEMBER
user: login: toyamarinyon
created_at: 2025-05-15T00:44:12Z
body: @shige I apologize for misunderstanding your intent earlier. I now see that your comment "I approved them, but I couldn't tell the difference" was referring to the difficulty in distinguishing between "PRs with incomplete functionality" versus "PRs with working functionality but architectural concerns."

I believe architectural quality remains important even in feature-flagged implementations for several reasons:

1. Future extensibility: Good architecture facilitates adding new features later
2. Technical debt: "It works for now" code increases long-term maintenance costs
3. Developer experience: Creates barriers for other developers extending the codebase

A balanced approach might be to either conduct architectural reviews before removing feature flags, or explicitly mark implementations as experimental (e.g., with `experimental_` prefix). While "if it works, it works" might be fine short-term, maintaining minimum architectural standards ensures long-term sustainability.


author_association: CONTRIBUTOR
user: login: shige
created_at: 2025-05-15T00:48:23Z
body: @toyamarinyon 

Thank you for your comment!

I regret that I was not able to express what I wanted to say clearly in my writing. 🙏 

> A balanced approach might be to either conduct architectural reviews before removing feature flags, or explicitly mark implementations as experimental (e.g., with `experimental_` prefix).

I think they are good approach! 👍 

author_association: MEMBER
user: login: satococoa
created_at: 2025-05-15T01:04:02Z
body: I've created a refactoring issue: https://github.com/giselles-ai/giselle/issues/850
I'll go ahead with merging this.
author_association: CONTRIBUTOR