Repository: giselles-ai/giselle
Pull request #920: feat(web-search): initial release of @giselle-sdk/web-search
Author: login: shige
Created: 2025-05-22T07:24:02Z | Merged: 2025-05-23T01:18:51Z

## Overview
This PR introduces a new package `@giselle-sdk/web-search` that provides functionality for scraping and extracting web page content in both HTML and Markdown formats. The package is designed to be used internally within the Giselle monorepo.

## Features
- Web page scraping with content extraction in HTML and Markdown formats
- Two provider implementations:
  - **Firecrawl**: Integration with [@mendable/firecrawl-js](https://www.npmjs.com/package/@mendable/firecrawl-js) API
  - **SelfMade**: Custom implementation using native fetch and Turndown for HTML-to-Markdown conversion
- Type-safe API implementation using zod

## Implementation Details
- TypeScript-based implementation
- Comprehensive test suite using Vitest
- Environment variable configuration for API keys
- Build and development scripts included

## Notes
- This package is currently not in use
- It will be utilized in a subsequent PR for implementing WebPagesNode functionality

## Testing
- Unit tests are included
- Firecrawl API tests can be run with `FIRECRAWL_API_KEY` and `VITEST_WITH_EXTERNAL_API=1`

## Related
- This PR is a prerequisite for the upcoming WebPagesNode implementation

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Introduced a new web search package enabling scraping and extraction of web page content in HTML and Markdown formats.
  - Supports two providers: one using an external API (requires API key) and another using native fetch with HTML-to-Markdown conversion (no API key needed).
  - Provides type-safe interfaces and schema validation for provider selection and scrape results.
  - Added unified websearch tool interface for consistent content fetching across providers.

- **Documentation**
  - Added comprehensive usage instructions, API details, environment setup, and development commands in the new package's README.

- **Tests**
  - Added test suites to validate scraping functionality and error handling for both providers.

- **Chores**
  - Added configuration files for TypeScript, build tools, and workspace management.
  - Updated package and dependency versions in workspace configuration files.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->


Diff:
diff --git a/docs/packages-license.md b/docs/packages-license.md
index d988f4a02b..e782be2a8f 100644
--- a/docs/packages-license.md
+++ b/docs/packages-license.md
@@ -12986,7 +12986,7 @@ Unknown manually approved
 
 
 <a name="tinyglobby"></a>
-### tinyglobby v0.2.10
+### tinyglobby v0.2.13
 #### 
 
 ##### Paths
diff --git a/packages/web-search/README.md b/packages/web-search/README.md
new file mode 100644
index 0000000000..9fa2be335a
--- /dev/null
+++ b/packages/web-search/README.md
@@ -0,0 +1,78 @@
+# @giselle-sdk/web-search
+
+A TypeScript library for scraping and extracting web page content as HTML and Markdown. Supports multiple providers, including Firecrawl (API-based) and a self-made (fetch + Turndown) implementation.
+
+## Features
+- Scrape web pages and extract content as HTML and/or Markdown
+- Two providers:
+  - **Firecrawl**: Uses the [@mendable/firecrawl-js](https://www.npmjs.com/package/@mendable/firecrawl-js) API
+  - **SelfMade**: Uses native fetch and [Turndown](https://www.npmjs.com/package/turndown) for HTML-to-Markdown conversion
+- Type-safe API using [zod](https://www.npmjs.com/package/zod)
+
+## Installation
+
+This package is intended for internal use within the Giselle monorepo. If you want to use it elsewhere, install dependencies:
+
+```sh
+pnpm install
+```
+
+## Usage
+
+### Unified Interface
+
+Create a web search tool by specifying the provider. Firecrawl requires an API key.
+
+```ts
+import { websearch } from "@giselle-sdk/web-search";
+
+const firecrawlTool = websearch({ provider: "firecrawl", apiKey: "fc-..." });
+const selfMadeTool = websearch({ provider: "self-made" });
+
+const result = await firecrawlTool.fetchUrl("https://example.com", ["html"]);
+console.log(result.html); // HTML content
+```
+
+Provider specific functions `firecrawlScrapeUrl` and `selfMadeScrapeUrl` are still exported if needed.
+
+## API
+
+Both providers return an object:
+
+```ts
+{
+  url: string;
+  title: string;
+  html: string;
+  markdown: string;
+}
+```
+
+- The `formats` argument is an array: `["html", "markdown"]`, `["html"]`, or `["markdown"]`.
+
+## Environment Variables
+
+- `FIRECRAWL_API_KEY`: Required for Firecrawl provider.
+- `VITEST_WITH_EXTERNAL_API=1`: (For testing) Enables tests that call the Firecrawl API.
+
+## Development
+
+- **Build:** `pnpm build`
+- **Type Check:** `pnpm check-types`
+- **Format:** `pnpm format`
+- **Test:** `pnpm test`
+- **Clean:** `pnpm clean`
+
+## Testing
+
+Uses [Vitest](https://vitest.dev/):
+
+```sh
+pnpm test
+```
+
+To run tests that require the Firecrawl API, set the environment variable:
+
+```sh
+FIRECRAWL_API_KEY=your-key VITEST_WITH_EXTERNAL_API=1 pnpm test
+```
diff --git a/packages/web-search/package.json b/packages/web-search/package.json
new file mode 100644
index 0000000000..7ffa362207
--- /dev/null
+++ b/packages/web-search/package.json
@@ -0,0 +1,36 @@
+{
+	"name": "@giselle-sdk/web-search",
+	"version": "0.0.0",
+	"private": true,
+	"main": "./dist/index.js",
+	"module": "./dist/index.mjs",
+	"types": "./dist/index.d.ts",
+	"scripts": {
+		"build": "tsup",
+		"check-types": "tsc --noEmit",
+		"clean": "rm -rf .turbo dist node_modules",
+		"format": "biome check --write .",
+		"test": "vitest run"
+	},
+	"exports": {
+		"./package.json": "./package.json",
+		".": {
+			"types": "./dist/index.d.ts",
+			"import": "./dist/index.mjs",
+			"require": "./dist/index.js",
+			"default": "./dist/index.js"
+		}
+	},
+	"devDependencies": {
+		"@giselle/giselle-sdk-tsconfig": "workspace:*",
+		"@types/node": "catalog:",
+		"@types/turndown": "catalog:",
+		"tsup": "catalog:",
+		"vitest": "catalog:"
+	},
+	"dependencies": {
+		"@mendable/firecrawl-js": "catalog:",
+		"turndown": "catalog:",
+		"zod": "catalog:"
+	}
+}
diff --git a/packages/web-search/src/firecrawl.test.ts b/packages/web-search/src/firecrawl.test.ts
new file mode 100644
index 0000000000..c19fb12ccf
--- /dev/null
+++ b/packages/web-search/src/firecrawl.test.ts
@@ -0,0 +1,56 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url", apiKey)).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return html (markdown empty)",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, apiKey, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(typeof result.html).toBe("string");
+			expect(result.markdown).toBe("");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+
+	it(
+		"should scrape a valid URL and return markdown (html empty)",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, apiKey, ["markdown"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(result.html).toBe("");
+			expect(typeof result.markdown).toBe("string");
+			expect(result.markdown.length).toBeGreaterThan(0);
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+
+	it(
+		"should scrape a valid URL and return both html and markdown",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, apiKey, ["html", "markdown"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(typeof result.html).toBe("string");
+			expect(typeof result.markdown).toBe("string");
+			expect(result.html.length).toBeGreaterThan(0);
+			expect(result.markdown.length).toBeGreaterThan(0);
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+});
diff --git a/packages/web-search/src/firecrawl.ts b/packages/web-search/src/firecrawl.ts
new file mode 100644
index 0000000000..06d54e653a
--- /dev/null
+++ b/packages/web-search/src/firecrawl.ts
@@ -0,0 +1,60 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+export const ALLOWED_SCRAPE_FORMATS = ["markdown", "html"] as const;
+
+export type AllowedScrapeFormats = (typeof ALLOWED_SCRAPE_FORMATS)[number];
+
+export type FirecrawlScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+export async function scrapeUrl(
+	url: string,
+	apiKey: string,
+	formats: AllowedScrapeFormats[] = [...ALLOWED_SCRAPE_FORMATS],
+): Promise<FirecrawlScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const firecrawlApp = new FirecrawlApp({ apiKey });
+	const timeoutMs = 10000;
+	const timeoutPromise: Promise<never> = new Promise((_, reject) =>
+		setTimeout(
+			() => reject(new Error("Request timed out after 10 seconds")),
+			timeoutMs,
+		),
+	);
+	const response = (await Promise.race([
+		firecrawlApp.scrapeUrl(url, { formats: formats }),
+		timeoutPromise,
+	])) as ScrapeResponse | ErrorResponse;
+	if (!response.success) {
+		throw new Error(`Failed to scrape: ${(response as ErrorResponse).error}`);
+	}
+	const doc = response as ScrapeResponse;
+	return {
+		url,
+		title: doc.metadata?.title || "",
+		html: doc.html || "",
+		markdown: doc.markdown || "",
+	};
+}
diff --git a/packages/web-search/src/index.ts b/packages/web-search/src/index.ts
new file mode 100644
index 0000000000..7efc989964
--- /dev/null
+++ b/packages/web-search/src/index.ts
@@ -0,0 +1,19 @@
+import { z } from "zod";
+import { FirecrawlWebSearchProvider, firecrawlProviderName } from "./firecrawl";
+import { SelfMadeWebSearchProvider, selfMadeProviderName } from "./self-made";
+export { websearch } from "./websearch";
+
+export { scrapeUrl as firecrawlScrapeUrl } from "./firecrawl";
+export { scrapeUrl as selfMadeScrapeUrl } from "./self-made";
+
+export const WebSearchProviderSchema = z.union([
+	FirecrawlWebSearchProvider,
+	SelfMadeWebSearchProvider,
+]);
+
+export type WebSearchProvider = z.infer<typeof WebSearchProviderSchema>;
+
+export const WebSearchProviders = z.enum([
+	firecrawlProviderName,
+	selfMadeProviderName,
+]);
diff --git a/packages/web-search/src/self-made.test.ts b/packages/web-search/src/self-made.test.ts
new file mode 100644
index 0000000000..275bd78f18
--- /dev/null
+++ b/packages/web-search/src/self-made.test.ts
@@ -0,0 +1,42 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./self-made";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run network-dependent tests if VITEST_WITH_EXTERNAL_API is set
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it("should fetch a valid URL and return html (markdown empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(result.markdown).toBe("");
+	});
+
+	it("should fetch a valid URL and return markdown (html empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(result.html).toBe("");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+
+	it("should fetch a valid URL and return both html and markdown", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html", "markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.html.length).toBeGreaterThan(0);
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+});
diff --git a/packages/web-search/src/self-made.ts b/packages/web-search/src/self-made.ts
new file mode 100644
index 0000000000..3dcd1927ac
--- /dev/null
+++ b/packages/web-search/src/self-made.ts
@@ -0,0 +1,66 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+/**
+ * Scrapes the content of a given URL and returns the result in the specified formats.
+ *
+ * @param {string} url - The URL to scrape. Must be a valid URL.
+ * @param {("html" | "markdown")[]} [formats=["html"]] - An optional array specifying the desired output formats.
+ *        - "html": Includes the raw HTML content.
+ *        - "markdown": Includes the content converted to Markdown.
+ * @returns {Promise<SelfMadeScrapeResult>} A promise resolving to an object containing the scraped URL, title, HTML, and/or Markdown.
+ * @throws {Error} If the URL is invalid or if the fetch operation fails.
+ */
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
+	const html = await res.text();
+	// Extract title from HTML
+	const match = html.match(/<title>(.*?)<\/title>/is);
+	const title = match ? match[1].trim() : "";
+
+	let markdown = "";
+	if (formats.includes("markdown")) {
+		markdown = turndownService.turndown(html);
+	}
+
+	return {
+		url,
+		title,
+		html: formats.includes("html") ? html : "",
+		markdown: formats.includes("markdown") ? markdown : "",
+	};
+}
diff --git a/packages/web-search/src/websearch.test.ts b/packages/web-search/src/websearch.test.ts
new file mode 100644
index 0000000000..0cc415ceef
--- /dev/null
+++ b/packages/web-search/src/websearch.test.ts
@@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
diff --git a/packages/web-search/src/websearch.ts b/packages/web-search/src/websearch.ts
new file mode 100644
index 0000000000..60c6a0380c
--- /dev/null
+++ b/packages/web-search/src/websearch.ts
@@ -0,0 +1,50 @@
+import {
+	firecrawlProviderName,
+	scrapeUrl as firecrawlScrapeUrl,
+} from "./firecrawl";
+import {
+	type selfMadeProviderName,
+	scrapeUrl as selfMadeScrapeUrl,
+} from "./self-made";
+
+export type AllowedFormats = "html" | "markdown";
+
+export interface WebsearchConfigFirecrawl {
+	provider: typeof firecrawlProviderName;
+	apiKey?: string;
+}
+
+export interface WebsearchConfigSelfMade {
+	provider: typeof selfMadeProviderName;
+}
+
+export type WebsearchConfig =
+	| WebsearchConfigFirecrawl
+	| WebsearchConfigSelfMade;
+
+export type WebSearchResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+export interface WebsearchTool {
+	fetchUrl: (
+		url: string,
+		formats?: AllowedFormats[],
+	) => Promise<WebSearchResult>;
+}
+
+export function websearch(config: WebsearchConfig): WebsearchTool {
+	if (config.provider === firecrawlProviderName) {
+		return {
+			fetchUrl: (url: string, formats?: AllowedFormats[]) =>
+				firecrawlScrapeUrl(url, config.apiKey ?? "", formats),
+		};
+	}
+	return {
+		fetchUrl: (url: string, formats?: AllowedFormats[]) =>
+			selfMadeScrapeUrl(url, formats),
+	};
+}
diff --git a/packages/web-search/tsconfig.json b/packages/web-search/tsconfig.json
new file mode 100644
index 0000000000..141aa6680e
--- /dev/null
+++ b/packages/web-search/tsconfig.json
@@ -0,0 +1,9 @@
+{
+	"extends": "../../tools/tsconfig/node.json",
+	"compilerOptions": {
+		"outDir": "./dist",
+		"rootDir": "./src"
+	},
+	"include": ["src/**/*.ts"],
+	"exclude": ["node_modules", "dist"]
+}
diff --git a/packages/web-search/tsup.config.ts b/packages/web-search/tsup.config.ts
new file mode 100644
index 0000000000..9c04aaf0f6
--- /dev/null
+++ b/packages/web-search/tsup.config.ts
@@ -0,0 +1,10 @@
+import { defineConfig } from "tsup";
+
+export default defineConfig({
+	entry: ["src/index.ts"],
+	dts: true,
+	format: ["esm", "cjs"],
+	splitting: false,
+	sourcemap: true,
+	clean: true,
+});
diff --git a/packages/web-search/turbo.json b/packages/web-search/turbo.json
new file mode 100644
index 0000000000..d190fa51a0
--- /dev/null
+++ b/packages/web-search/turbo.json
@@ -0,0 +1,9 @@
+{
+	"extends": ["//"],
+	"tasks": {
+		"build": {
+			"outputs": ["**/dist/**"]
+		},
+		"test": {}
+	}
+}
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index d2555d09cb..e9951cb460 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -36,6 +36,9 @@ catalogs:
     '@graphql-codegen/client-preset':
       specifier: 4.7.0
       version: 4.7.0
+    '@mendable/firecrawl-js':
+      specifier: 1.25.1
+      version: 1.25.1
     '@octokit/auth-app':
       specifier: 7.1.1
       version: 7.1.1
@@ -1266,6 +1269,34 @@ importers:
         specifier: 'catalog:'
         version: 8.3.5(jiti@2.4.2)(postcss@8.5.1)(typescript@5.7.3)(yaml@2.7.0)
 
+  packages/web-search:
+    dependencies:
+      '@mendable/firecrawl-js':
+        specifier: 'catalog:'
+        version: 1.25.1
+      turndown:
+        specifier: 'catalog:'
+        version: 7.2.0
+      zod:
+        specifier: 'catalog:'
+        version: 3.24.1
+    devDependencies:
+      '@giselle/giselle-sdk-tsconfig':
+        specifier: workspace:*
+        version: link:../../tools/tsconfig
+      '@types/node':
+        specifier: 'catalog:'
+        version: 22.14.0
+      '@types/turndown':
+        specifier: 'catalog:'
+        version: 5.0.5
+      tsup:
+        specifier: 'catalog:'
+        version: 8.3.5(jiti@2.4.2)(postcss@8.5.1)(typescript@5.7.3)(yaml@2.7.0)
+      vitest:
+        specifier: 'catalog:'
+        version: 3.1.4(@types/debug@4.1.12)(@types/node@22.14.0)(jiti@2.4.2)(jsdom@26.0.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(lightningcss@1.29.1)(terser@5.39.0)(yaml@2.7.0)
+
   packages/workflow-designer:
     dependencies:
       '@giselle-sdk/data-type':
@@ -2578,6 +2609,10 @@ packages:
   '@manypkg/get-packages@1.1.3':
     resolution: {integrity: sha512-fo+QhuU3qE/2TQMQmbVMqaQ6EWbMhi4ABWP+O4AM1NqPBuy0OrApV5LO6BrrgnhtAHS2NH6RrVk9OL181tTi8A==}
 
+  '@mendable/firecrawl-js@1.25.1':
+    resolution: {integrity: sha512-i4MyViBXV9l4Z/gaY1tWXs74ZD4Oiea7xKUOzmYmC/AYKWEMrUuOhZHvdNJCOTHkTWq81xe4pYBQaGp+rtDPuA==}
+    engines: {node: '>=22.0.0'}
+
   '@mendable/firecrawl-js@1.6.1':
     resolution: {integrity: sha512-mpgNCxmlRMmnN2XgECYmodn4zQlDhCgZuXeDwfkQtY4jYQBwF3ChQYDfL4HYsQL9/DmP/5169HUlR/g8UBS6og==}
 
@@ -8658,10 +8693,6 @@ packages:
   tinyexec@0.3.2:
     resolution: {integrity: sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==}
 
-  tinyglobby@0.2.10:
-    resolution: {integrity: sha512-Zc+8eJlFMvgatPZTl6A9L/yht8QqdmUNtURHaKZLmKBE12hNPSrqNkUp2cs3M/UKmNVVAMFQYSjYIVHDjW5zew==}
-    engines: {node: '>=12.0.0'}
-
   tinyglobby@0.2.13:
     resolution: {integrity: sha512-mEwzpUgrLySlveBwEVDMKk5B57bhLPYovRfPAXD5gA/98Opn0rCDj3GtLwFvCvH5RK9uPCExUROW5NjDwvqkxw==}
     engines: {node: '>=12.0.0'}
@@ -10733,6 +10764,15 @@ snapshots:
       globby: 11.1.0
       read-yaml-file: 1.1.0
 
+  '@mendable/firecrawl-js@1.25.1':
+    dependencies:
+      axios: 1.7.9
+      typescript-event-target: 1.1.1
+      zod: 3.24.1
+      zod-to-json-schema: 3.24.1(zod@3.24.1)
+    transitivePeerDependencies:
+      - debug
+
   '@mendable/firecrawl-js@1.6.1(ws@8.18.1(bufferutil@4.0.9)(utf-8-validate@6.0.5))':
     dependencies:
       axios: 1.7.9
@@ -13394,7 +13434,7 @@ snapshots:
 
   '@types/pg@8.11.13':
     dependencies:
-      '@types/node': 22.14.0
+      '@types/node': 22.15.3
       pg-protocol: 1.8.0
       pg-types: 4.0.2
 
@@ -13873,7 +13913,7 @@ snapshots:
   axios@1.7.9:
     dependencies:
       follow-redirects: 1.15.9
-      form-data: 4.0.1
+      form-data: 4.0.2
       proxy-from-env: 1.1.0
     transitivePeerDependencies:
       - debug
@@ -13973,7 +14013,6 @@ snapshots:
     dependencies:
       es-errors: 1.3.0
       function-bind: 1.1.2
-    optional: true
 
   call-bound@1.0.3:
     dependencies:
@@ -14505,7 +14544,6 @@ snapshots:
       get-intrinsic: 1.3.0
       has-tostringtag: 1.0.2
       hasown: 2.0.2
-    optional: true
 
   esbuild-register@3.6.0(esbuild@0.19.12):
     dependencies:
@@ -14778,7 +14816,6 @@ snapshots:
       combined-stream: 1.0.8
       es-set-tostringtag: 2.1.0
       mime-types: 2.1.35
-    optional: true
 
   formdata-node@4.4.1:
     dependencies:
@@ -14880,7 +14917,6 @@ snapshots:
       has-symbols: 1.1.0
       hasown: 2.0.2
       math-intrinsics: 1.1.0
-    optional: true
 
   get-nonce@1.0.1: {}
 
@@ -15093,7 +15129,6 @@ snapshots:
   has-tostringtag@1.0.2:
     dependencies:
       has-symbols: 1.1.0
-    optional: true
 
   hasown@2.0.2:
     dependencies:
@@ -17703,11 +17738,6 @@ snapshots:
 
   tinyexec@0.3.2: {}
 
-  tinyglobby@0.2.10:
-    dependencies:
-      fdir: 6.4.3(picomatch@4.0.2)
-      picomatch: 4.0.2
-
   tinyglobby@0.2.13:
     dependencies:
       fdir: 6.4.4(picomatch@4.0.2)
@@ -17825,7 +17855,7 @@ snapshots:
       source-map: 0.8.0-beta.0
       sucrase: 3.35.0
       tinyexec: 0.3.2
-      tinyglobby: 0.2.10
+      tinyglobby: 0.2.13
       tree-kill: 1.2.2
     optionalDependencies:
       postcss: 8.5.1
diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml
index 756ca6379c..3d4c3ab1cb 100644
--- a/pnpm-workspace.yaml
+++ b/pnpm-workspace.yaml
@@ -12,6 +12,7 @@ catalog:
   "@turbo/gen": 2.3.3
   next: 15.3.0
   unstorage: 1.14.4
+  "@mendable/firecrawl-js": 1.25.1
   ai: 4.3.9
   "@types/node": 22.14.0
   "@types/react": 19.1.0


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

Initial release of the `@giselle-sdk/web-search` package, adding web scraping and content extraction in HTML/Markdown formats with two providers and full TypeScript setup.

- Configure monorepo workspace, build, and compiler settings for the new package
- Implement two providers (`self-made` and Firecrawl) with Zod schemas and unit tests
- Add package manifest, README documentation, and bump dependency license info

### Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 4 comments.

<details>
<summary>Show a summary per file</summary>

| File                               | Description                                           |
| ---------------------------------- | ----------------------------------------------------- |
| pnpm-workspace.yaml                | Register new dependency `@mendable/firecrawl-js`      |
| packages/web-search/turbo.json     | Define build and test tasks                          |
| packages/web-search/tsup.config.ts | Configure bundling outputs (esm, cjs)                 |
| packages/web-search/tsconfig.json  | Set TypeScript compiler options                      |
| packages/web-search/src/self-made.ts       | Implement native fetch + Turndown provider           |
| packages/web-search/src/firecrawl.ts        | Implement Firecrawl API provider with timeout       |
| packages/web-search/src/index.ts           | Export unified schemas, types, and provider Zod enum |
| packages/web-search/src/self-made.test.ts  | Add tests for invalid/valid URL scraping (self-made)|
| packages/web-search/src/firecrawl.test.ts  | Add tests (skipped by default) for Firecrawl provider|
| packages/web-search/package.json           | Define package setup, scripts, dependencies         |
| packages/web-search/README.md               | Document usage, API, environment variables          |
| docs/packages-license.md            | Bump `tinyglobby` version in license docs            |
</details>


<details>
<summary>Files not reviewed (1)</summary>

* **pnpm-lock.yaml**: Language not supported
</details>

<details>
<summary>Comments suppressed due to low confidence (2)</summary>

**packages/web-search/src/self-made.test.ts:12**
* The tests for `scrapeUrl` do not verify the extracted `title` field. Consider adding assertions to cover title extraction logic.
```
describe("scrapeUrl (valid URL)", () => {
```
**packages/web-search/src/self-made.ts:4**
* [nitpick] The provider name `"self-made"` uses a hyphen, while the Firecrawl provider is named `"firecrawl"`. Consider using a consistent naming convention across providers.
```
export const selfMadeProviderName = "self-made" as const;
```
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:19:00Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 7**

<details>
<summary>🧹 Nitpick comments (15)</summary><blockquote>

<details>
<summary>pnpm-workspace.yaml (1)</summary><blockquote>

`15-15`: **LGTM! Consider adding a comment for improved maintainability.**

Adding the `@mendable/firecrawl-js` dependency is appropriate for supporting the new web scraping functionality in the Firecrawl provider implementation.


Consider adding a comment above or beside this line to indicate its purpose (e.g., `# Required for web-search package's Firecrawl provider`). This would improve maintainability and help other developers understand why this dependency is needed.

</blockquote></details>
<details>
<summary>packages/web-search/src/firecrawl.test.ts (1)</summary><blockquote>

`4-5`: **Consider documenting timeout value choice.**

The 20-second timeout for API tests is reasonable, but adding a brief comment explaining this specific value would improve code maintainability.

```diff
 const TEST_URL = "https://example.com/";
-const HEAVY_TEST_TIMEOUT = 20000;
+// 20 seconds timeout for external API calls, which may be slow or rate-limited
+const HEAVY_TEST_TIMEOUT = 20000;
```

</blockquote></details>
<details>
<summary>packages/web-search/src/self-made.test.ts (1)</summary><blockquote>

`6-10`: **Consider adding more specific error message tests.**

While you test that invalid URLs throw an error, consider also verifying the specific error message content to ensure it's helpful for debugging.

```diff
-	it("should throw on invalid URL", async () => {
-		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
-	});
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow("Invalid URL: not-a-url");
+	});
```

</blockquote></details>
<details>
<summary>packages/web-search/README.md (3)</summary><blockquote>

`61-65`: **Fix formatting in Environment Variables section.**

The static analysis tool detected a punctuation issue in this section. Consider adding consistent formatting to the environment variables list.

```diff
## Environment Variables

-  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider.
-  - `VITEST_WITH_EXTERNAL_API=1`: (For testing) Enables tests that call the Firecrawl API.
+- `FIRECRAWL_API_KEY`: Required for Firecrawl provider.
+- `VITEST_WITH_EXTERNAL_API=1`: (For testing) Enables tests that call the Firecrawl API.
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~63-~63: Loose punctuation mark.
Context: ...ronment Variables  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider. - `VIT...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

---

`12-18`: **Enhance installation section with specific instructions.**

The installation section could be more specific about what exactly needs to be installed and how to integrate the package within or outside the monorepo.

```diff
## Installation

This package is intended for internal use within the Giselle monorepo. If you want to use it elsewhere, install dependencies:

```sh
-pnpm install
+# Within the monorepo
+pnpm install
+
+# If using outside the monorepo
+pnpm add @giselle-sdk/web-search @mendable/firecrawl-js turndown zod
```

---

`46-57`: **Consider adding error handling examples to the API documentation.**

The API section could benefit from examples of error handling, explaining what errors might be thrown and how to handle them.

```diff
## API

Both providers return an object:

```ts
{
  url: string;
  title: string;
  html: string;
  markdown: string;
}
```

- The `formats` argument is an array: `["html", "markdown"]`, `["html"]`, or `["markdown"]`.
+
+### Error Handling
+
+Both providers may throw errors in these scenarios:
+
+```ts
+try {
+  const result = await selfMadeScrapeUrl("https://example.com");
+} catch (error) {
+  // Handle invalid URLs: "Invalid URL: ..."
+  // Handle failed requests: "Failed to fetch: 404 Not Found"
+  // Handle timeouts: "Request timed out after ... ms"
+}
+```
```

</blockquote></details>
<details>
<summary>packages/web-search/package.json (2)</summary><blockquote>

`14-23`: **Consider adding test:watch script for development.**

Adding a watch mode for tests would be helpful during development to automatically run tests when files change.

```diff
"scripts": {
  "build": "tsup",
  "check-types": "tsc --noEmit",
  "clean": "rm -rf .turbo dist node_modules",
  "format": "biome check --write .",
- "test": "vitest run"
+ "test": "vitest run",
+ "test:watch": "vitest"
},
```

---

`1-36`: **Consider adding a files field to control package contents.**

Even for private packages, it's good practice to explicitly define which files should be included in the package distribution.

```diff
{
	"name": "@giselle-sdk/web-search",
	"version": "0.0.0",
	"private": true,
	"main": "./dist/index.js",
	"module": "./dist/index.mjs",
	"types": "./dist/index.d.ts",
+	"files": [
+		"dist",
+		"README.md"
+	],
	"scripts": {
		// ...
	},
```

</blockquote></details>
<details>
<summary>packages/web-search/src/index.ts (2)</summary><blockquote>

`5-6`: **Consider adding a unified API or factory function.**

Currently, consumers need to explicitly choose which provider to use. Consider adding a factory function or unified API that allows switching providers more easily.

```diff
export { scrapeUrl as firecrawlScrapeUrl } from "./firecrawl";
export { scrapeUrl as selfMadeScrapeUrl } from "./self-made";

+/**
+ * Create a scraper based on the specified provider
+ */
+export function createScraper(provider: z.infer<typeof WebSearchProviders>) {
+  switch (provider) {
+    case "firecrawl":
+      return { scrapeUrl: firecrawlScrapeUrl };
+    case "self-made":
+      return { scrapeUrl: selfMadeScrapeUrl };
+    default:
+      throw new Error(`Unknown provider: ${provider}`);
+  }
+}
```

---

`8-18`: **Document the provider schemas and enum.**

Add JSDoc comments to the exported schemas and enum to make their purpose clearer to consumers.

```diff
+/**
+ * Schema for validating web search providers
+ */
export const WebSearchProviderSchema = z.union([
	FirecrawlWebSearchProvider,
	SelfMadeWebSearchProvider,
]);

export type WebSearchProvider = z.infer<typeof WebSearchProviderSchema>;

+/**
+ * Enum of available web search provider names
+ */
export const WebSearchProviders = z.enum([
	firecrawlProviderName,
	selfMadeProviderName,
]);
```

</blockquote></details>
<details>
<summary>packages/web-search/src/self-made.ts (1)</summary><blockquote>

`27-35`: **Consider enhancing URL validation**

While the current URL validation will catch malformed URLs, consider using a more robust validation approach for edge cases.

```diff
-	try {
-		new URL(url);
-	} catch {
-		throw new Error(`Invalid URL: ${url}`);
-	}
+	try {
+		const parsedUrl = new URL(url);
+		// Additional validation: ensure protocol is http or https
+		if (!['http:', 'https:'].includes(parsedUrl.protocol)) {
+			throw new Error(`Unsupported protocol: ${parsedUrl.protocol}`);
+		}
+	} catch (error) {
+		throw new Error(`Invalid URL: ${url} - ${(error as Error).message}`);
+	}
```

</blockquote></details>
<details>
<summary>packages/web-search/src/firecrawl.ts (4)</summary><blockquote>

`31-39`: **URL validation is consistent with self-made provider**

Good job maintaining consistency in URL validation between providers. As with the self-made provider, consider enhancing the validation to check for supported protocols.


Same enhancement recommended for the self-made provider could be applied here for protocol validation.

---

`47-53`: **Improve type safety in Promise.race**

The type assertion on the Promise.race result could be improved for better type safety.

```diff
-	const response = (await Promise.race([
-		firecrawlApp.scrapeUrl(url, { formats: formats }),
-		timeoutPromise,
-	])) as ScrapeResponse | ErrorResponse;
+	const response: ScrapeResponse | ErrorResponse = await Promise.race([
+		firecrawlApp.scrapeUrl(url, { formats: formats }),
+		timeoutPromise,
+	]);
```

---

`51-53`: **Consider more specific error handling**

The error handling could be more specific to provide better debugging information.

```diff
-	if (!response.success) {
-		throw new Error(`Failed to scrape: ${(response as ErrorResponse).error}`);
-	}
+	if (!response.success) {
+		const errorResponse = response as ErrorResponse;
+		throw new Error(`Failed to scrape ${url}: ${errorResponse.error || 'Unknown error'}`);
+	}
```

---

`54-60`: **Consider handling missing fields more explicitly**

The current implementation uses logical OR to default to empty strings, which is concise but might mask potential issues.

```diff
-	return {
-		url,
-		title: doc.metadata?.title || "",
-		html: doc.html || "",
-		markdown: doc.markdown || "",
-	};
+	// Create result object with explicit defaults for missing fields
+	const result: FirecrawlScrapeResult = {
+		url,
+		title: "",
+		html: "",
+		markdown: "",
+	};
+	
+	// Conditionally populate fields from response when available
+	if (doc.metadata?.title) result.title = doc.metadata.title;
+	if (formats.includes("html") && doc.html) result.html = doc.html;
+	if (formats.includes("markdown") && doc.markdown) result.markdown = doc.markdown;
+	
+	return result;
```

Alternatively, a simpler approach with improved consistency:

```diff
-	return {
-		url,
-		title: doc.metadata?.title || "",
-		html: doc.html || "",
-		markdown: doc.markdown || "",
-	};
+	return {
+		url,
+		title: doc.metadata?.title || "",
+		html: formats.includes("html") ? (doc.html || "") : "",
+		markdown: formats.includes("markdown") ? (doc.markdown || "") : "",
+	};
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**


<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 50c886a3fd4c9f17c36ae5ef6c99719dcbc851d8 and 0e37c2709986a8b8c430c49fe6d56ff032cf1c63.

</details>

<details>
<summary>⛔ Files ignored due to path filters (1)</summary>

* `pnpm-lock.yaml` is excluded by `!**/pnpm-lock.yaml`

</details>

<details>
<summary>📒 Files selected for processing (12)</summary>

* `docs/packages-license.md` (1 hunks)
* `packages/web-search/README.md` (1 hunks)
* `packages/web-search/package.json` (1 hunks)
* `packages/web-search/src/firecrawl.test.ts` (1 hunks)
* `packages/web-search/src/firecrawl.ts` (1 hunks)
* `packages/web-search/src/index.ts` (1 hunks)
* `packages/web-search/src/self-made.test.ts` (1 hunks)
* `packages/web-search/src/self-made.ts` (1 hunks)
* `packages/web-search/tsconfig.json` (1 hunks)
* `packages/web-search/tsup.config.ts` (1 hunks)
* `packages/web-search/turbo.json` (1 hunks)
* `pnpm-workspace.yaml` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧬 Code Graph Analysis (4)</summary>

<details>
<summary>packages/web-search/src/self-made.test.ts (2)</summary><blockquote>

<details>
<summary>packages/web-search/src/index.ts (2)</summary>

* `scrapeUrl` (5-5)
* `scrapeUrl` (6-6)

</details>
<details>
<summary>packages/web-search/src/self-made.ts (1)</summary>

* `scrapeUrl` (27-56)

</details>

</blockquote></details>
<details>
<summary>packages/web-search/src/index.ts (2)</summary><blockquote>

<details>
<summary>packages/web-search/src/firecrawl.ts (3)</summary>

* `FirecrawlWebSearchProvider` (9-11)
* `FirecrawlWebSearchProvider` (13-15)
* `firecrawlProviderName` (7-7)

</details>
<details>
<summary>packages/web-search/src/self-made.ts (3)</summary>

* `SelfMadeWebSearchProvider` (6-8)
* `SelfMadeWebSearchProvider` (10-12)
* `selfMadeProviderName` (4-4)

</details>

</blockquote></details>
<details>
<summary>packages/web-search/src/firecrawl.test.ts (3)</summary><blockquote>

<details>
<summary>packages/web-search/src/index.ts (2)</summary>

* `scrapeUrl` (5-5)
* `scrapeUrl` (6-6)

</details>
<details>
<summary>packages/web-search/src/firecrawl.ts (1)</summary>

* `scrapeUrl` (31-61)

</details>
<details>
<summary>packages/web-search/src/self-made.ts (1)</summary>

* `scrapeUrl` (27-56)

</details>

</blockquote></details>
<details>
<summary>packages/web-search/src/firecrawl.ts (2)</summary><blockquote>

<details>
<summary>packages/web-search/src/index.ts (2)</summary>

* `scrapeUrl` (5-5)
* `scrapeUrl` (6-6)

</details>
<details>
<summary>packages/web-search/src/self-made.ts (1)</summary>

* `scrapeUrl` (27-56)

</details>

</blockquote></details>

</details><details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>packages/web-search/README.md</summary>

[uncategorized] ~63-~63: Loose punctuation mark.
Context: ...ronment Variables  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider. - `VIT...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (12)</summary><blockquote>

<details>
<summary>docs/packages-license.md (1)</summary>

`12989-12989`: <details>
<summary>✅ Verification successful</summary>

**Validate the Tinyglobby version bump**  
Please verify that `tinyglobby@0.2.13` is published on npm and that its license remains MIT as listed.

---

🏁 Script executed:

```shell
#!/usr/bin/env bash
# Check the published version and license of tinyglobby
npm view tinyglobby version
npm view tinyglobby license
```

Length of output: 70

---



</details>

**Tinyglobby Version and License Confirmed**  
Verified that `tinyglobby@0.2.13` is published on npm and retains the MIT license.

</details>
<details>
<summary>packages/web-search/tsconfig.json (1)</summary>

`1-9`: **Clean and well-structured TypeScript configuration.**

The TypeScript configuration follows best practices by extending a base configuration, clearly defining output and source directories, and setting appropriate include/exclude patterns.

</details>
<details>
<summary>packages/web-search/turbo.json (1)</summary>

`1-9`: **Appropriate Turborepo configuration.**

The configuration properly extends the base configuration and correctly defines the build and test tasks with appropriate output settings.

</details>
<details>
<summary>packages/web-search/tsup.config.ts (1)</summary>

`1-10`: **Well-configured build setup.**

The tsup configuration covers all essential build requirements including dual module format support (ESM/CJS), type declarations, source maps, and clean builds.

</details>
<details>
<summary>packages/web-search/src/self-made.test.ts (1)</summary>

`1-40`: **Good test coverage of core functionality.**

The tests are well-structured and cover the essential functionality of the `scrapeUrl` function, including handling invalid URLs and different output format combinations.

</details>
<details>
<summary>packages/web-search/README.md (1)</summary>

`1-87`: **Well-organized documentation with clear examples.**

The README provides comprehensive information about the package features, installation, usage, API, environment variables, and development process. The examples are clear and illustrate both providers effectively.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~63-~63: Loose punctuation mark.
Context: ...ronment Variables  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider. - `VIT...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

</details>
<details>
<summary>packages/web-search/package.json (1)</summary>

`1-36`: **Well-structured package configuration with appropriate dependencies.**

The package.json file follows modern package conventions and correctly defines entry points, scripts, and dependencies for both CommonJS and ES modules.

</details>
<details>
<summary>packages/web-search/src/index.ts (1)</summary>

`1-19`: **Clean API with good type safety.**

The exports are well-organized, with clear naming that differentiates between providers. The Zod schemas and TypeScript types provide good type safety.

</details>
<details>
<summary>packages/web-search/src/self-made.ts (3)</summary>

`4-8`: **Good constant definition and schema setup**

Using a constant for the provider name with a literal type in the schema is a clean approach for type safety and consistency throughout the codebase.

---

`21-25`: **Well-configured TurndownService**

The TurndownService configuration explicitly sets consistent Markdown styling options, which is good for predictable output formatting.

---

`50-55`: **Clear and consistent return object structure**

The return object structure is well-designed, conditionally including content based on requested formats. This approach is consistent with the Firecrawl implementation.

</details>
<details>
<summary>packages/web-search/src/firecrawl.ts (1)</summary>

`40-50`: **Well-implemented timeout handling**

The timeout implementation using Promise.race is a good approach. This ensures requests don't hang indefinitely.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:24:07Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:26:04Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:37:03Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:38:15Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:39:01Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:39:21Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:39:41Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:40:11Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:40:59Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:41:27Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:42:09Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:42:27Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:42:56Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T08:43:18Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T08:43:32Z
user: login: toyamarinyon
body: Overall, I think it looks good! There are some implementation adjustments I'd like to make, so I'll create a pull request - please wait.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T10:02:21Z
user: login: toyamarinyon
body: I've created Pull Request #929 with the changes I was concerned about. I think this Pull Request is good to merge as is, so please take a look after it's merged.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T12:10:11Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 1**

<details>
<summary>♻️ Duplicate comments (2)</summary><blockquote>

<details>
<summary>packages/web-search/src/firecrawl.ts (1)</summary><blockquote>

`28-37`: **Consider adding input validation for the API key parameter.**

The function accepts an API key parameter but doesn't validate whether it's empty or not before using it. This could lead to confusing errors when calling the Firecrawl API.



I see this issue was already identified in previous reviews and will be addressed in a separate issue (#925).

</blockquote></details>
<details>
<summary>packages/web-search/src/websearch.ts (1)</summary><blockquote>

`39-44`: **Handle missing API key more gracefully.**

The code currently falls back to an empty string if the API key is not provided, which might cause issues with the Firecrawl API.



I see this issue was already identified in previous reviews and will be addressed in a separate issue (#925).

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (7)</summary><blockquote>

<details>
<summary>packages/web-search/src/firecrawl.ts (3)</summary><blockquote>

`39-49`: **Consider extracting the timeout logic into a reusable utility function.**

The timeout implementation with Promise.race is good, but it might be worth extracting this pattern into a reusable utility function since this pattern could be useful elsewhere in the codebase.


```diff
+// In a separate utility file
+export function withTimeout<T>(promise: Promise<T>, timeoutMs: number, errorMessage: string): Promise<T> {
+  const timeoutPromise: Promise<never> = new Promise((_, reject) =>
+    setTimeout(() => reject(new Error(errorMessage)), timeoutMs)
+  );
+  return Promise.race([promise, timeoutPromise]) as Promise<T>;
+}

// In firecrawl.ts
-const timeoutMs = 10000;
-const timeoutPromise: Promise<never> = new Promise((_, reject) =>
-  setTimeout(
-    () => reject(new Error("Request timed out after 10 seconds")),
-    timeoutMs,
-  ),
-);
-const response = (await Promise.race([
-  firecrawlApp.scrapeUrl(url, { formats: formats }),
-  timeoutPromise,
-])) as ScrapeResponse | ErrorResponse;
+const response = await withTimeout(
+  firecrawlApp.scrapeUrl(url, { formats: formats }),
+  10000,
+  "Request timed out after 10 seconds"
+) as ScrapeResponse | ErrorResponse;
```

---

`50-53`: **Improve error handling specificity.**

The current error handling is appropriate but could be more specific to help with debugging. Consider adding more context about what might have caused the error.


```diff
 if (!response.success) {
-  throw new Error(`Failed to scrape: ${(response as ErrorResponse).error}`);
+  const errorResponse = response as ErrorResponse;
+  throw new Error(`Failed to scrape URL "${url}": ${errorResponse.error}`);
 }
```

---

`54-59`: **Add documentation for return value fallbacks.**

The function uses fallbacks for missing values in the response, which is good defensive programming. Consider adding a comment to explain this behavior for future maintainers.


```diff
 const doc = response as ScrapeResponse;
+// Ensure all properties have fallback values in case the API response is incomplete
 return {
   url,
   title: doc.metadata?.title || "",
   html: doc.html || "",
   markdown: doc.markdown || "",
 };
```

</blockquote></details>
<details>
<summary>packages/web-search/src/websearch.test.ts (1)</summary><blockquote>

`15-25`: **Consider adding more test coverage for the websearch tool.**

The test is correctly conditional based on the environment variable, but it only checks if the result has an "html" property of type string. Consider adding more assertions to verify the structure and content of the result.


```diff
 it("should fetch a valid URL", async () => {
   const result = await tool.fetchUrl(TEST_URL, ["html"]);
   expect(result).toHaveProperty("html");
   expect(typeof result.html).toBe("string");
+  expect(result).toHaveProperty("url", TEST_URL);
+  expect(result).toHaveProperty("title");
+  expect(typeof result.title).toBe("string");
+  expect(result.html.length).toBeGreaterThan(0);
 });
```

</blockquote></details>
<details>
<summary>packages/web-search/README.md (2)</summary><blockquote>

`36-36`: **Fix hyphenation in "Provider-specific".**

When used as a modifier, "Provider-specific" should be hyphenated.


```diff
-Provider specific functions `firecrawlScrapeUrl` and `selfMadeScrapeUrl` are still exported if needed.
+Provider-specific functions `firecrawlScrapeUrl` and `selfMadeScrapeUrl` are still exported if needed.
```

---

`38-50`: **Add error handling examples to the API documentation.**

The API documentation covers the response structure but doesn't explain how to handle errors. Consider adding examples of error handling to help users implement robust code.


```diff
 ## API

 Both providers return an object:

 ```ts
 {
   url: string;
   title: string;
   html: string;
   markdown: string;
 }
 ```

 - The `formats` argument is an array: `["html", "markdown"]`, `["html"]`, or `["markdown"]`.
+
+### Error Handling
+
+The library throws errors for invalid URLs, timeouts, and failed scraping attempts:
+
+```ts
+try {
+  const result = await tool.fetchUrl("https://example.com");
+  // Use result...
+} catch (error) {
+  console.error("Scraping failed:", error.message);
+  // Handle specific error types...
+}
+```
```

</blockquote></details>
<details>
<summary>packages/web-search/src/websearch.ts (1)</summary><blockquote>

`39-50`: **Consider adding validation for the config object.**

The websearch function doesn't validate the config object before using it. Consider adding validation to ensure the config object is well-formed.


```diff
+import { z } from "zod";
+
+const FirecrawlConfig = z.object({
+  provider: z.literal(firecrawlProviderName),
+  apiKey: z.string().optional(),
+});
+
+const SelfMadeConfig = z.object({
+  provider: z.literal(selfMadeProviderName),
+});
+
+const Config = z.union([FirecrawlConfig, SelfMadeConfig]);

 export function websearch(config: WebsearchConfig): WebsearchTool {
+  const validatedConfig = Config.parse(config);
+  
-  if (config.provider === firecrawlProviderName) {
+  if (validatedConfig.provider === firecrawlProviderName) {
     return {
       fetchUrl: (url: string, formats?: AllowedFormats[]) =>
-        firecrawlScrapeUrl(url, config.apiKey ?? "", formats),
+        firecrawlScrapeUrl(url, validatedConfig.apiKey ?? "", formats),
     };
   }
   return {
     fetchUrl: (url: string, formats?: AllowedFormats[]) =>
       selfMadeScrapeUrl(url, formats),
   };
 }
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**


<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a and d4a4c145554b9836c1618f9eac4ae57fd9a87395.

</details>

<details>
<summary>📒 Files selected for processing (7)</summary>

* `packages/web-search/README.md` (1 hunks)
* `packages/web-search/src/firecrawl.test.ts` (1 hunks)
* `packages/web-search/src/firecrawl.ts` (1 hunks)
* `packages/web-search/src/index.ts` (1 hunks)
* `packages/web-search/src/self-made.test.ts` (1 hunks)
* `packages/web-search/src/websearch.test.ts` (1 hunks)
* `packages/web-search/src/websearch.ts` (1 hunks)

</details>

<details>
<summary>🚧 Files skipped from review as they are similar to previous changes (3)</summary>

* packages/web-search/src/firecrawl.test.ts
* packages/web-search/src/self-made.test.ts
* packages/web-search/src/index.ts

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>packages/web-search/README.md</summary>

[uncategorized] ~35-~35: When ‘Provider-specific’ is used as a modifier, it is usually spelled with a hyphen.
Context: ....log(result.html); // HTML content ```  Provider specific functions `firecrawlScrapeUrl` and `sel...

(SPECIFIC_HYPHEN)

---

[uncategorized] ~55-~55: Loose punctuation mark.
Context: ...ronment Variables  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider. - `VIT...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

</details>

<details>
<summary>⏰ Context from checks skipped due to timeout of 90000ms (2)</summary>

* GitHub Check: codeql / languages (javascript) / Perform CodeQL for javascript
* GitHub Check: check

</details>

<details>
<summary>🔇 Additional comments (3)</summary><blockquote>

<details>
<summary>packages/web-search/src/websearch.test.ts (1)</summary>

`8-13`: **LGTM! Good error handling test.**

The test properly verifies that the websearch tool throws an error when given an invalid URL.

</details>
<details>
<summary>packages/web-search/README.md (1)</summary>

`53-57`: **Great documentation of environment variables!**

Clear explanation of the required environment variables for both the Firecrawl provider and testing.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~55-~55: Loose punctuation mark.
Context: ...ronment Variables  - `FIRECRAWL_API_KEY`: Required for Firecrawl provider. - `VIT...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

</details>
<details>
<summary>packages/web-search/src/websearch.ts (1)</summary>

`12-15`: **Good separation of provider-specific configuration interfaces.**

The separation of provider-specific configuration interfaces makes the code more maintainable and type-safe.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T01:11:09Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-23T01:11:49Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T01:12:18Z
user: login: shige
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-23T01:16:21Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T01:16:41Z

Review comments:
diff_hunk: @@ -0,0 +1,61 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
path: packages/web-search/src/firecrawl.ts
user: login: Copilot
body: The API key is defaulting to an empty string if `FIRECRAWL_API_KEY` is not set, which may lead to confusing errors from the Firecrawl API. Consider validating the key early and throwing a clear error if missing.
```suggestion
if (!process.env.FIRECRAWL_API_KEY) {
    throw new Error("Environment variable FIRECRAWL_API_KEY is required but not set.");
}
const apiKey = process.env.FIRECRAWL_API_KEY;
```
created_at: 2025-05-22T08:18:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
path: packages/web-search/src/self-made.ts
user: login: Copilot
body: [nitpick] URL validation and error handling logic is duplicated in both providers. Consider extracting these into a shared utility to reduce code duplication.
created_at: 2025-05-22T08:18:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
path: packages/web-search/src/self-made.ts
user: login: Copilot
body: [nitpick] Consider adding a JSDoc comment for `scrapeUrl` detailing parameters, return type, and potential errors to improve code documentation.
```suggestion

/**
 * Scrapes the content of a given URL and returns the result in the specified formats.
 *
 * @param {string} url - The URL to scrape. Must be a valid URL.
 * @param {("html" | "markdown")[]} [formats=["html"]] - An optional array specifying the desired output formats.
 *        - "html": Includes the raw HTML content.
 *        - "markdown": Includes the content converted to Markdown.
 * @returns {Promise<SelfMadeScrapeResult>} A promise resolving to an object containing the scraped URL, title, HTML, and/or Markdown.
 * @throws {Error} If the URL is invalid or if the fetch operation fails.
 */
```
created_at: 2025-05-22T08:19:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,36 @@
+{
+	"name": "@giselle-sdk/web-search",
+	"version": "0.0.0",
+	"private": true,
+	"main": "./dist/index.js",
+	"module": "./dist/index.mjs",
+	"types": "./dist/index.d.ts",
+	"scripts": {
+		"build": "tsup",
+		"check-types": "tsc --noEmit",
+		"clean": "rm -rf .turbo dist node_modules",
path: packages/web-search/package.json
user: login: Copilot
body: [nitpick] The `clean` script uses shell commands that may not be cross-platform compatible. Consider using a Node.js package like `rimraf` or `del` for consistent behavior.
created_at: 2025-05-22T08:19:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Consolidate duplicate markdown-only tests.**

This test and the one at lines 41-52 both test the same functionality (scraping with only markdown format) but have different assertions. The second test is more thorough, checking that HTML is empty when only markdown is requested.

```diff
- it(
-   "should scrape a valid URL and return markdown and html",
-   async () => {
-     const result = await scrapeUrl(TEST_URL, ["markdown"]);
-     expect(result).toHaveProperty("markdown");
-     expect(result).toHaveProperty("html");
-     expect(typeof result.markdown).toBe("string");
-     expect(typeof result.html).toBe("string");
-   },
-   HEAVY_TEST_TIMEOUT,
- );
```

Keep the more comprehensive test at lines 41-52 which properly verifies that HTML is empty when only markdown is requested.


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion

```

</details>

<!-- suggestion_end -->

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/firecrawl.test.ts between lines 17 and 27, there is a
duplicate test for scraping a URL with only markdown format. Remove this test
and keep the more comprehensive test at lines 41-52, which correctly asserts
that the HTML output is empty when only markdown is requested. This
consolidation avoids redundant tests and ensures thorough validation.
```

</details>

<!-- This is an auto-generated comment by CodeRabbit -->

✅ Addressed in commit 6ee63a3
created_at: 2025-05-22T08:24:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+
+	it(
+		"should scrape a valid URL and return html (markdown empty)",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(typeof result.html).toBe("string");
+			expect(result.markdown).toBe("");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Enhance HTML-only test to verify content length.**

While the test correctly verifies that markdown is empty when only HTML is requested, it should also verify that HTML content has a length greater than zero for consistency with other tests.

```diff
  it(
    "should scrape a valid URL and return html (markdown empty)",
    async () => {
      const result = await scrapeUrl(TEST_URL, ["html"]);
      expect(result).toHaveProperty("html");
      expect(result).toHaveProperty("markdown");
      expect(typeof result.html).toBe("string");
      expect(result.markdown).toBe("");
+     expect(result.html.length).toBeGreaterThan(0);
    },
    HEAVY_TEST_TIMEOUT,
  );
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
	it(
		"should scrape a valid URL and return html (markdown empty)",
		async () => {
			const result = await scrapeUrl(TEST_URL, ["html"]);
			expect(result).toHaveProperty("html");
			expect(result).toHaveProperty("markdown");
			expect(typeof result.html).toBe("string");
			expect(result.markdown).toBe("");
			expect(result.html.length).toBeGreaterThan(0);
		},
		HEAVY_TEST_TIMEOUT,
	);
```

</details>

<!-- suggestion_end -->

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/firecrawl.test.ts around lines 29 to 39, the test for
scraping a valid URL with only HTML requested checks that the markdown is empty
but does not verify the HTML content length. Enhance this test by adding an
assertion to confirm that the HTML string length is greater than zero, ensuring
the HTML content is not empty and improving test robustness.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:24:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,39 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./self-made";
+
+const TEST_URL = "https://example.com/";
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+describe("scrapeUrl (valid URL)", () => {
+	it("should fetch a valid URL and return html (markdown empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(result.markdown).toBe("");
+	});
+
+	it("should fetch a valid URL and return markdown (html empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(result.html).toBe("");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+
+	it("should fetch a valid URL and return both html and markdown", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html", "markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.html.length).toBeGreaterThan(0);
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+});
path: packages/web-search/src/self-made.test.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add tests for HTTP error handling.**

There are no tests for how the function handles HTTP errors (e.g., 404, 500). Add tests to verify that appropriate error messages are thrown when fetch fails.

```ts
it("should throw when fetch returns non-OK status", async () => {
  // Override the mock for this specific test
  global.fetch = vi.fn().mockResolvedValue({
    ok: false,
    status: 404,
    statusText: "Not Found"
  });
  
  await expect(scrapeUrl(TEST_URL)).rejects.toThrow("Failed to fetch: 404 Not Found");
});
```

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/self-made.test.ts around lines 12 to 39, add new test
cases to cover HTTP error handling for the scrapeUrl function. Specifically,
create tests that mock fetch to return non-OK responses like 404 or 500 status
codes and verify that scrapeUrl throws appropriate error messages indicating the
failure status. Use mocking to override fetch behavior within these tests and
assert that the function rejects with the expected error message.
```

</details>

---

_🛠️ Refactor suggestion_

**Consider using mocks to avoid external network dependencies.**

The current tests make actual network calls to example.com, which could make tests brittle if the external site changes or is unavailable. Consider using a mocking library like `vi.mock` to mock the fetch responses.

```ts
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { scrapeUrl } from "./self-made";

const TEST_URL = "https://example.com/";
const MOCK_HTML = "<html><head><title>Example Domain</title></head><body><h1>Example Domain</h1></body></html>";

beforeEach(() => {
  // Mock global fetch
  global.fetch = vi.fn().mockResolvedValue({
    ok: true,
    text: () => Promise.resolve(MOCK_HTML),
    status: 200,
    statusText: "OK"
  });
});

afterEach(() => {
  vi.restoreAllMocks();
});

// ... rest of your tests
```

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/self-made.test.ts around lines 12 to 39, the tests
currently perform real network requests which can cause flakiness. To fix this,
mock the global fetch function using vi.mock or vi.fn in a beforeEach hook to
return a fixed HTML response, and restore mocks after each test. This will
isolate tests from external dependencies and improve reliability.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:24:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
path: packages/web-search/src/self-made.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add timeout handling for fetch requests**

The current implementation doesn't have a timeout for fetch requests, which could lead to hanging connections for unresponsive servers.

```diff
-	const res = await fetch(url);
+	const controller = new AbortController();
+	const timeoutId = setTimeout(() => controller.abort(), 10000); // 10-second timeout
+	
+	try {
+		const res = await fetch(url, { 
+			signal: controller.signal 
+		});
+		clearTimeout(timeoutId);
+		
+		if (!res.ok) {
+			throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+		}
+		
+		// Check if content type is HTML
+		const contentType = res.headers.get('content-type');
+		if (!contentType || !contentType.includes('text/html')) {
+			throw new Error(`Expected HTML content but got: ${contentType}`);
+		}
+		
+		const html = await res.text();
+	} catch (error) {
+		if (error instanceof DOMException && error.name === 'AbortError') {
+			throw new Error('Request timed out after 10 seconds');
+		}
+		throw error;
+	}
```


> Committable suggestion skipped: line range outside the PR's diff.

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/self-made.ts around lines 36 to 39, the fetch request
lacks timeout handling, which can cause the request to hang indefinitely if the
server is unresponsive. Modify the fetch call to include a timeout mechanism,
such as using an AbortController to abort the fetch after a specified timeout
duration. Ensure that the timeout triggers an error or rejection to handle the
case where the fetch takes too long.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:24:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
+	const html = await res.text();
+	// Extract title from HTML
+	const match = html.match(/<title>(.*?)<\/title>/is);
+	const title = match ? match[1].trim() : "";
path: packages/web-search/src/self-made.ts
user: login: coderabbitai[bot]
body: _💡 Verification agent_

<details>
<summary>🧩 Analysis chain</summary>

**Improve title extraction reliability**

Using regex for HTML parsing can be fragile. Consider using a proper HTML parser like cheerio or the DOM API.

```diff
-	// Extract title from HTML
-	const match = html.match(/<title>(.*?)<\/title>/is);
-	const title = match ? match[1].trim() : "";
+	// Extract title using DOM parser for more reliable parsing
+	let title = "";
+	if (typeof DOMParser !== 'undefined') {
+		// Browser environment
+		const parser = new DOMParser();
+		const doc = parser.parseFromString(html, 'text/html');
+		title = doc.title || "";
+	} else {
+		// Node.js environment - fallback to regex with additional checks
+		const match = html.match(/<title>(.*?)<\/title>/is);
+		title = match ? match[1].trim() : "";
+	}
```

Alternatively, consider adding a dependency on a lightweight HTML parser like cheerio:

```typescript
import * as cheerio from 'cheerio';

// Then replace title extraction with:
const $ = cheerio.load(html);
const title = $('title').text().trim() || "";
```

---

🌐 Web query:

```web
Is cheerio or jsdom better for HTML parsing in Node.js?
```

💡 Result:

When choosing between Cheerio and jsdom for HTML parsing in Node.js, the decision largely depends on your specific use case, particularly the complexity of the HTML and whether JavaScript execution or browser-like environment is needed.

## Cheerio

- **Designed for speed and efficiency:** Cheerio is lightweight and focuses solely on parsing and manipulating static HTML. It mimics the jQuery syntax, making it very accessible to developers familiar with jQuery[1][2][3][8].
- **Performance:** Cheerio is considerably faster and more resource-efficient than jsdom, often cited as being several times faster in typical scraping tasks. It consumes less memory because it does not simulate a full browser environment and does not execute JavaScript[3][4][6].
- **Use cases:** Best suited for web scraping, data extraction, and scenarios where only HTML parsing and simple DOM manipulation are required without JavaScript execution or dynamic content rendering[2][3][6].
- **Limitations:** Does not support JavaScript execution, so it cannot handle dynamic web pages that require JS rendering. It only manipulates a static DOM tree derived from the raw HTML[3].

## jsdom

- **Full DOM environment:** jsdom provides a near-complete, standards-compliant DOM API that mimics a browser environment, including support for JavaScript execution, event handling, and complex DOM interactions[1][2][6].
- **Use cases:** Perfect for testing front-end code, server-side rendering, or any application needing to simulate browser behavior in Node.js. Also useful for parsing when scripts on the page need to be executed to generate the DOM[1][2].
- **Performance:** Slower and more memory-intensive than Cheerio because it creates a full virtual browser environment, including `window` and `document` objects[3][6].
- **Complex web pages:** Better suited for parsing complex or dynamic HTML documents generated or modified by JavaScript.

## Summary Comparison

| Feature                   | Cheerio                                | jsdom                                   |
|---------------------------|---------------------------------------|-----------------------------------------|
| Parsing focus             | Static HTML parsing                    | Full DOM simulation with JS execution   |
| API style                 | jQuery-like                           | Browser-standard DOM API                 |
| JavaScript execution      | No                                    | Yes                                     |
| Performance               | Fast, lightweight, low memory usage   | Slower, higher memory usage              |
| Use case recommendation   | Web scraping, simple DOM manipulation | Testing, server-side rendering, complex DOM manipulation |
| Handling dynamic pages    | Not suitable                         | Suitable                               |

**In brief:** If your project involves mostly static HTML parsing and you prioritize speed and resource efficiency, Cheerio is the better choice. If you need to parse pages that rely on JavaScript to build the DOM or require a full DOM API, jsdom is more appropriate despite its higher overhead[1][2][3][6][8].

Citations:

- 1: https://npm-compare.com/cheerio,domino,jsdom,parse5
- 2: https://scrapeops.io/nodejs-web-scraping-playbook/best-nodejs-html-parsing-libraries/
- 3: https://app.studyraid.com/en/read/12482/403533/comparing-cheerio-with-other-parsing-libraries
- 4: https://github.com/cheeriojs/cheerio/issues/700
- 5: https://www.zenrows.com/alternative/cheerio
- 6: https://npm-compare.com/cheerio,jsdom,node-html-parser,parse5
- 7: https://npm-compare.com/cheerio,html,htmlparser2,jsdom
- 8: https://dev.to/apiforseo/best-html-parsing-libraries-in-javascript-6bn

---



</details>

**Switch to Cheerio for Title Extraction**

Using a regex to pull out `<title>` can easily break on malformed or minified HTML. Since this package runs in Node.js and only needs static parsing, Cheerio is the fastest, most reliable choice.

• File: packages/web-search/src/self-made.ts  
• Lines: 41–43  

Replace your regex block with:

```diff
--- a/packages/web-search/src/self-made.ts
+++ b/packages/web-search/src/self-made.ts
@@ -41,7 +41,9 @@
-	// Extract title from HTML
-	const match = html.match(/<title>(.*?)<\/title>/is);
-	const title = match ? match[1].trim() : "";
+	// Extract title using Cheerio for reliable HTML parsing
+	import cheerio from 'cheerio';
+
+	const $ = cheerio.load(html);
+	const title = $('title').text().trim() || "";
```

If you ever need to support browser environments, you can guard against the absence of `DOMParser`:

```ts
let title = "";
if (typeof DOMParser !== "undefined") {
  const doc = new DOMParser().parseFromString(html, "text/html");
  title = doc.title.trim();
} else {
  import cheerio from "cheerio";
  const $ = cheerio.load(html);
  title = $("title").text().trim();
}
```

This removes the fragility of regex, avoids the heavy overhead of jsdom, and keeps your dependency footprint minimal.

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/self-made.ts around lines 41 to 43, replace the
regex-based title extraction with Cheerio to improve reliability. Import
cheerio, load the HTML string using cheerio.load, then extract and trim the
title text using $('title').text().trim(). This avoids fragile regex parsing and
is more robust for static HTML in Node.js environments.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:24:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,61 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
+const firecrawlApp = new FirecrawlApp({ apiKey });
path: packages/web-search/src/firecrawl.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Handle missing API key more gracefully**

The code uses an empty string as a fallback when the API key is not provided, but this will likely cause issues when making API calls.

```diff
-const apiKey = process.env.FIRECRAWL_API_KEY || "";
-const firecrawlApp = new FirecrawlApp({ apiKey });
+const apiKey = process.env.FIRECRAWL_API_KEY;
+if (!apiKey) {
+  console.warn('Warning: FIRECRAWL_API_KEY environment variable not set. Firecrawl provider will not work properly.');
+}
+const firecrawlApp = new FirecrawlApp({ apiKey: apiKey || "" });
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
const apiKey = process.env.FIRECRAWL_API_KEY;
if (!apiKey) {
  console.warn('Warning: FIRECRAWL_API_KEY environment variable not set. Firecrawl provider will not work properly.');
}
const firecrawlApp = new FirecrawlApp({ apiKey: apiKey || "" });
```

</details>

<!-- suggestion_end -->

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/firecrawl.ts around lines 17 to 18, the code
currently falls back to an empty string if the FIRECRAWL_API_KEY environment
variable is missing, which can cause issues during API calls. Instead, add a
check to verify if the API key is present; if not, throw a clear error or handle
the missing key scenario gracefully before initializing FirecrawlApp. This
ensures the application does not proceed with an invalid or empty API key.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:24:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
path: packages/web-search/src/self-made.ts
user: login: shige
body: added: e2b49fc5835de1595c3be53fdc934bbd5254940e
created_at: 2025-05-22T08:26:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: shige
body: fix: 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a
created_at: 2025-05-22T08:37:03Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: coderabbitai[bot]
body: <details>
<summary>🧩 Analysis chain</summary>

🏁 Script executed:
```
#!/bin/bash
# Fetch the specific commit to see what changes were made
git show 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a --stat
git show 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a
```

Length of output: 1632

---



</details>

@shige I see you've implemented the suggestion to remove the duplicate test case. The commit successfully eliminates the redundant test while preserving the more comprehensive test that properly verifies both the presence of markdown content and the absence of HTML content when only markdown format is requested. Nice cleanup! 🧹

ʕ •ᴥ•ʔ

---

<details>
<summary>🧬 Code Graph Analysis Results</summary>

### Relevant snippets for understanding `packages/web-search/src/firecrawl.test.ts`

---

#### From `packages/web-search/src/firecrawl.ts` (lines 31-61):

```ts
export async function scrapeUrl(
	url: string,
	formats: AllowedScrapeFormats[] = [...ALLOWED_SCRAPE_FORMATS],
): Promise<FirecrawlScrapeResult> {
	try {
		new URL(url);
	} catch {
		throw new Error(`Invalid URL: ${url}`);
	}
	const timeoutMs = 10000;
	const timeoutPromise: Promise<never> = new Promise((_, reject) =>
		setTimeout(
			() => reject(new Error("Request timed out after 10 seconds")),
			timeoutMs,
		),
	);
	const response = (await Promise.race([
		firecrawlApp.scrapeUrl(url, { formats: formats }),
		timeoutPromise,
	])) as ScrapeResponse | ErrorResponse;
	if (!response.success) {
		throw new Error(`Failed to scrape: ${(response as ErrorResponse).error}`);
	}
	const doc = response as ScrapeResponse;
	return {
		url,
		title: doc.metadata?.title || "",
		html: doc.html || "",
		markdown: doc.markdown || "",
	};
}
```

**Summary:**

- `scrapeUrl` function takes a URL string and an optional array of formats (`"html"` or `"markdown"`).
- Validates the URL, throws if invalid.
- Uses a 10-second timeout to race against the scraping operation.
- Calls `firecrawlApp.scrapeUrl` with the URL and formats.
- Throws if the scrape response indicates failure.
- Returns an object containing the URL, title (from metadata), HTML content, and markdown content.
- HTML and markdown fields default to empty strings if not present.

---

#### From `packages/web-search/src/self-made.ts` (lines 37-66):

```ts
export async function scrapeUrl(
	url: string,
	formats: ("html" | "markdown")[] = ["html"],
): Promise<SelfMadeScrapeResult> {
	try {
		new URL(url);
	} catch {
		throw new Error(`Invalid URL: ${url}`);
	}
	const res = await fetch(url);
	if (!res.ok) {
		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
	}
	const html = await res.text();
	// Extract title from HTML
	const match = html.match(/<title>(.*?)<\/title>/is);
	const title = match ? match[1].trim() : "";

	let markdown = "";
	if (formats.includes("markdown")) {
		markdown = turndownService.turndown(html);
	}

	return {
		url,
		title,
		html: formats.includes("html") ? html : "",
		markdown: formats.includes("markdown") ? markdown : "",
	};
}
```

**Summary:**

- Another `scrapeUrl` function variant that fetches the URL directly.
- Validates URL, throws if invalid.
- Fetches the HTML content, throws if fetch fails.
- Extracts the page title from the HTML `<title>` tag.
- Converts HTML to markdown if requested using `turndownService`.
- Returns an object with URL, title, HTML (or empty string), and markdown (or empty string).

---

#### From `packages/web-search/src/index.ts` (lines 5-6):

```ts
scrapeUrl as firecrawlScrapeUrl
scrapeUrl as selfMadeScrapeUrl
```

**Summary:**

- The `scrapeUrl` function is exported under two aliases: `firecrawlScrapeUrl` and `selfMadeScrapeUrl`.
- This indicates the project has two scraping implementations, one in `firecrawl.ts` and one in `self-made.ts`.

---

These snippets provide the implementation context for the `scrapeUrl` function tested in `firecrawl.test.ts`. The test file imports `scrapeUrl` from `firecrawl.ts` and tests its behavior on invalid URLs and valid URLs with different format requests (`html`, `markdown`, or both). The timeout and error handling in the implementation are relevant to the test timeouts and error expectations.

</details>


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer><details>
<summary>🧩 Analysis chain</summary>

🏁 Script executed:
```
#!/bin/bash
# Fetch the specific commit to see what changes were made
git show 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a --stat
git show 6ee63a3a64cd659ebc40adc1c3a7fcd47db3d70a
```

Length of output: 1632

---



</details>

@shige I see you've implemented the suggestion to remove the duplicate test case. The commit successfully eliminates the redundant test while preserving the more comprehensive test that properly verifies both the presence of markdown content and the absence of HTML content when only markdown format is requested. Nice cleanup! 🧹

ʕ •ᴥ•ʔ

---

<details>
<summary>🧬 Code Graph Analysis Results</summary>

### Relevant snippets for understanding `packages/web-search/src/firecrawl.test.ts`

---

#### From `packages/web-search/src/firecrawl.ts` (lines 31-61):

```ts
export async function scrapeUrl(
	url: string,
	formats: AllowedScrapeFormats[] = [...ALLOWED_SCRAPE_FORMATS],
): Promise<FirecrawlScrapeResult> {
	try {
		new URL(url);
	} catch {
		throw new Error(`Invalid URL: ${url}`);
	}
	const timeoutMs = 10000;
	const timeoutPromise: Promise<never> = new Promise((_, reject) =>
		setTimeout(
			() => reject(new Error("Request timed out after 10 seconds")),
			timeoutMs,
		),
	);
	const response = (await Promise.race([
		firecrawlApp.scrapeUrl(url, { formats: formats }),
		timeoutPromise,
	])) as ScrapeResponse | ErrorResponse;
	if (!response.success) {
		throw new Error(`Failed to scrape: ${(response as ErrorResponse).error}`);
	}
	const doc = response as ScrapeResponse;
	return {
		url,
		title: doc.metadata?.title || "",
		html: doc.html || "",
		markdown: doc.markdown || "",
	};
}
```

**Summary:**

- `scrapeUrl` function takes a URL string and an optional array of formats (`"html"` or `"markdown"`).
- Validates the URL, throws if invalid.
- Uses a 10-second timeout to race against the scraping operation.
- Calls `firecrawlApp.scrapeUrl` with the URL and formats.
- Throws if the scrape response indicates failure.
- Returns an object containing the URL, title (from metadata), HTML content, and markdown content.
- HTML and markdown fields default to empty strings if not present.

---

#### From `packages/web-search/src/self-made.ts` (lines 37-66):

```ts
export async function scrapeUrl(
	url: string,
	formats: ("html" | "markdown")[] = ["html"],
): Promise<SelfMadeScrapeResult> {
	try {
		new URL(url);
	} catch {
		throw new Error(`Invalid URL: ${url}`);
	}
	const res = await fetch(url);
	if (!res.ok) {
		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
	}
	const html = await res.text();
	// Extract title from HTML
	const match = html.match(/<title>(.*?)<\/title>/is);
	const title = match ? match[1].trim() : "";

	let markdown = "";
	if (formats.includes("markdown")) {
		markdown = turndownService.turndown(html);
	}

	return {
		url,
		title,
		html: formats.includes("html") ? html : "",
		markdown: formats.includes("markdown") ? markdown : "",
	};
}
```

**Summary:**

- Another `scrapeUrl` function variant that fetches the URL directly.
- Validates URL, throws if invalid.
- Fetches the HTML content, throws if fetch fails.
- Extracts the page title from the HTML `<title>` tag.
- Converts HTML to markdown if requested using `turndownService`.
- Returns an object with URL, title, HTML (or empty string), and markdown (or empty string).

---

#### From `packages/web-search/src/index.ts` (lines 5-6):

```ts
scrapeUrl as firecrawlScrapeUrl
scrapeUrl as selfMadeScrapeUrl
```

**Summary:**

- The `scrapeUrl` function is exported under two aliases: `firecrawlScrapeUrl` and `selfMadeScrapeUrl`.
- This indicates the project has two scraping implementations, one in `firecrawl.ts` and one in `self-made.ts`.

---

These snippets provide the implementation context for the `scrapeUrl` function tested in `firecrawl.test.ts`. The test file imports `scrapeUrl` from `firecrawl.ts` and tests its behavior on invalid URLs and valid URLs with different format requests (`html`, `markdown`, or both). The timeout and error handling in the implementation are relevant to the test timeouts and error expectations.

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:38:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+
+	it(
+		"should scrape a valid URL and return html (markdown empty)",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(typeof result.html).toBe("string");
+			expect(result.markdown).toBe("");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: shige
body: I will do later https://github.com/giselles-ai/giselle/issues/922
created_at: 2025-05-22T08:39:01Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./firecrawl";
+
+const TEST_URL = "https://example.com/";
+const HEAVY_TEST_TIMEOUT = 20000;
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+// Only run external API tests if VITEST_WITH_EXTERNAL_API environment variable is set
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+(hasExternalApiEnv ? describe : describe.skip)("scrapeUrl (valid URL)", () => {
+	it(
+		"should scrape a valid URL and return markdown and html",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["markdown"]);
+			expect(result).toHaveProperty("markdown");
+			expect(result).toHaveProperty("html");
+			expect(typeof result.markdown).toBe("string");
+			expect(typeof result.html).toBe("string");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
+
+	it(
+		"should scrape a valid URL and return html (markdown empty)",
+		async () => {
+			const result = await scrapeUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(result).toHaveProperty("markdown");
+			expect(typeof result.html).toBe("string");
+			expect(result.markdown).toBe("");
+		},
+		HEAVY_TEST_TIMEOUT,
+	);
path: packages/web-search/src/firecrawl.test.ts
user: login: coderabbitai[bot]
body: @shige Thanks for creating a tracking issue for this enhancement! I've noted that you'll address this test improvement in issue #922 at a later time.


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige Thanks for creating a tracking issue for this enhancement! I've noted that you'll address this test improvement in issue #922 at a later time.</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:39:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,39 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./self-made";
+
+const TEST_URL = "https://example.com/";
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+describe("scrapeUrl (valid URL)", () => {
+	it("should fetch a valid URL and return html (markdown empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(result.markdown).toBe("");
+	});
+
+	it("should fetch a valid URL and return markdown (html empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(result.html).toBe("");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+
+	it("should fetch a valid URL and return both html and markdown", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html", "markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.html.length).toBeGreaterThan(0);
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+});
path: packages/web-search/src/self-made.test.ts
user: login: shige
body: I dont want to use mock.
created_at: 2025-05-22T08:39:40Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,39 @@
+import { describe, expect, it } from "vitest";
+import { scrapeUrl } from "./self-made";
+
+const TEST_URL = "https://example.com/";
+
+describe("scrapeUrl (invalid URL)", () => {
+	it("should throw on invalid URL", async () => {
+		await expect(scrapeUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+describe("scrapeUrl (valid URL)", () => {
+	it("should fetch a valid URL and return html (markdown empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(result.markdown).toBe("");
+	});
+
+	it("should fetch a valid URL and return markdown (html empty)", async () => {
+		const result = await scrapeUrl(TEST_URL, ["markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(result.html).toBe("");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+
+	it("should fetch a valid URL and return both html and markdown", async () => {
+		const result = await scrapeUrl(TEST_URL, ["html", "markdown"]);
+		expect(result).toHaveProperty("html");
+		expect(result).toHaveProperty("markdown");
+		expect(typeof result.html).toBe("string");
+		expect(typeof result.markdown).toBe("string");
+		expect(result.html.length).toBeGreaterThan(0);
+		expect(result.markdown.length).toBeGreaterThan(0);
+	});
+});
path: packages/web-search/src/self-made.test.ts
user: login: coderabbitai[bot]
body: @shige I understand and respect your preference not to use mocks in your tests. The current approach using actual network requests to example.com is valid, especially since it's a stable site designed for testing purposes. I'll keep this in mind for future suggestions.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/self-made.test.ts:12-39
Timestamp: 2025-05-22T08:40:10.174Z
Learning: For the web-search package tests, the developer prefers using real network requests rather than mocking HTTP calls.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige I understand and respect your preference not to use mocks in your tests. The current approach using actual network requests to example.com is valid, especially since it's a stable site designed for testing purposes. I'll keep this in mind for future suggestions.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/self-made.test.ts:12-39
Timestamp: 2025-05-22T08:40:10.174Z
Learning: For the web-search package tests, the developer prefers using real network requests rather than mocking HTTP calls.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:40:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
path: packages/web-search/src/self-made.ts
user: login: shige
body: I will do it in https://github.com/giselles-ai/giselle/issues/923
created_at: 2025-05-22T08:40:59Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
path: packages/web-search/src/self-made.ts
user: login: coderabbitai[bot]
body: @shige Understood! Thanks for creating a separate issue to track the timeout handling implementation. That's a good approach to keep this PR focused while ensuring the enhancement is properly addressed later.



(⌐■_■)

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/self-made.ts:46-49
Timestamp: 2025-05-22T08:41:26.571Z
Learning: The fetch requests in the web-search package will have timeout handling implemented in issue #923, addressing the potential issue of hanging connections for unresponsive servers.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige Understood! Thanks for creating a separate issue to track the timeout handling implementation. That's a good approach to keep this PR focused while ensuring the enhancement is properly addressed later.



(⌐■_■)

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/self-made.ts:46-49
Timestamp: 2025-05-22T08:41:26.571Z
Learning: The fetch requests in the web-search package will have timeout handling implemented in issue #923, addressing the potential issue of hanging connections for unresponsive servers.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:41:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
+	const html = await res.text();
+	// Extract title from HTML
+	const match = html.match(/<title>(.*?)<\/title>/is);
+	const title = match ? match[1].trim() : "";
path: packages/web-search/src/self-made.ts
user: login: shige
body: I will do it in https://github.com/giselles-ai/giselle/issues/924
created_at: 2025-05-22T08:42:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,56 @@
+import TurndownService from "turndown";
+import { z } from "zod";
+
+export const selfMadeProviderName = "self-made" as const;
+
+export const SelfMadeWebSearchProvider = z.object({
+	provider: z.literal(selfMadeProviderName),
+});
+
+export type SelfMadeWebSearchProvider = z.infer<
+	typeof SelfMadeWebSearchProvider
+>;
+
+export type SelfMadeScrapeResult = {
+	url: string;
+	title: string;
+	html: string;
+	markdown: string;
+};
+
+const turndownService = new TurndownService({
+	headingStyle: "atx", // Use # heading style for headings
+	codeBlockStyle: "fenced", // Use ``` fenced style for code blocks
+	bulletListMarker: "*", // Use * as the bullet list marker
+});
+
+export async function scrapeUrl(
+	url: string,
+	formats: ("html" | "markdown")[] = ["html"],
+): Promise<SelfMadeScrapeResult> {
+	try {
+		new URL(url);
+	} catch {
+		throw new Error(`Invalid URL: ${url}`);
+	}
+	const res = await fetch(url);
+	if (!res.ok) {
+		throw new Error(`Failed to fetch: ${res.status} ${res.statusText}`);
+	}
+	const html = await res.text();
+	// Extract title from HTML
+	const match = html.match(/<title>(.*?)<\/title>/is);
+	const title = match ? match[1].trim() : "";
path: packages/web-search/src/self-made.ts
user: login: coderabbitai[bot]
body: @shige Sounds good! Thanks for creating a tracking issue for the Cheerio implementation. This approach will make the HTML parsing more robust when you get to it.


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige Sounds good! Thanks for creating a tracking issue for the Cheerio implementation. This approach will make the HTML parsing more robust when you get to it.</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:42:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,61 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
+const firecrawlApp = new FirecrawlApp({ apiKey });
path: packages/web-search/src/firecrawl.ts
user: login: shige
body: I will do it in https://github.com/giselles-ai/giselle/issues/925
created_at: 2025-05-22T08:42:56Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,61 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
+const firecrawlApp = new FirecrawlApp({ apiKey });
path: packages/web-search/src/firecrawl.ts
user: login: coderabbitai[bot]
body: @shige Sounds good! I'll keep track of that issue for the API key handling improvement. Thanks for creating a separate issue to address it.


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige Sounds good! I'll keep track of that issue for the API key handling improvement. Thanks for creating a separate issue to address it.</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-22T08:43:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,61 @@
+import FirecrawlApp, {
+	type ErrorResponse,
+	type ScrapeResponse,
+} from "@mendable/firecrawl-js";
+import { z } from "zod";
+
+export const firecrawlProviderName = "firecrawl" as const;
+
+export const FirecrawlWebSearchProvider = z.object({
+	provider: z.literal(firecrawlProviderName),
+});
+
+export type FirecrawlWebSearchProvider = z.infer<
+	typeof FirecrawlWebSearchProvider
+>;
+
+const apiKey = process.env.FIRECRAWL_API_KEY || "";
path: packages/web-search/src/firecrawl.ts
user: login: shige
body: I will refactor in https://github.com/giselles-ai/giselle/issues/925
created_at: 2025-05-22T08:43:32Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
path: packages/web-search/src/websearch.test.ts
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add tests for the Firecrawl provider.**

Currently, the tests only cover the self-made provider. Consider adding tests for the Firecrawl provider with proper mocking to avoid external API calls during unit tests.


```typescript
import { vi } from "vitest";
import { websearch } from "./index";

// Mock the firecrawl module
vi.mock("./firecrawl", () => ({
  firecrawlProviderName: "firecrawl",
  scrapeUrl: vi.fn().mockResolvedValue({
    url: "https://example.com",
    title: "Example Domain",
    html: "<html>...</html>",
    markdown: "# Example Domain"
  })
}));

describe("websearch with firecrawl provider", () => {
  const tool = websearch({ provider: "firecrawl", apiKey: "test-key" });
  
  it("should fetch a valid URL", async () => {
    const result = await tool.fetchUrl("https://example.com", ["html"]);
    expect(result).toHaveProperty("html");
    expect(typeof result.html).toBe("string");
    expect(result).toHaveProperty("url", "https://example.com");
  });
});
```

<details>
<summary>🤖 Prompt for AI Agents</summary>

```
In packages/web-search/src/websearch.test.ts around lines 15 to 25, add a new
test suite for the Firecrawl provider to complement the existing self-made
provider tests. Mock the Firecrawl module using vitest's vi.mock to avoid real
API calls, providing a resolved value with expected properties like url, title,
html, and markdown. Then, create a describe block for the Firecrawl provider
that initializes the websearch tool with provider "firecrawl" and a test API
key, and add an it test case that calls fetchUrl with a valid URL, asserting the
presence and types of the returned properties including html and url.
```

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-23T01:11:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
path: packages/web-search/src/websearch.test.ts
user: login: shige
body: I do not want to use mock.
created_at: 2025-05-23T01:11:49Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
path: packages/web-search/src/websearch.test.ts
user: login: coderabbitai[bot]
body: @shige I understand you prefer not to use mocks. In that case, you could extend your current approach of using the environment variable flag to conditionally run the Firecrawl provider tests with real API calls:

```typescript
(hasExternalApiEnv ? describe : describe.skip)(
  "websearch firecrawl (valid URL)",
  () => {
    // Skip this test if no API key is provided
    const apiKey = process.env.FIRECRAWL_API_KEY;
    if (!apiKey) {
      it.skip("needs FIRECRAWL_API_KEY to run", () => {});
      return;
    }

    const tool = websearch({ provider: "firecrawl", apiKey });
    it("should fetch a valid URL", async () => {
      const result = await tool.fetchUrl(TEST_URL, ["html"]);
      expect(result).toHaveProperty("html");
      expect(typeof result.html).toBe("string");
    }, 20000); // Increased timeout for external API call
  },
);
```

This maintains your approach of using real API calls for thorough testing while keeping the tests skippable when appropriate.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/websearch.test.ts:15-25
Timestamp: 2025-05-23T01:12:16.998Z
Learning: When testing the web-search package, real API calls are preferred over mocks. Tests should use environment variables like VITEST_WITH_EXTERNAL_API and FIRECRAWL_API_KEY to conditionally run tests that require external services.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige I understand you prefer not to use mocks. In that case, you could extend your current approach of using the environment variable flag to conditionally run the Firecrawl provider tests with real API calls:

```typescript
(hasExternalApiEnv ? describe : describe.skip)(
  "websearch firecrawl (valid URL)",
  () => {
    // Skip this test if no API key is provided
    const apiKey = process.env.FIRECRAWL_API_KEY;
    if (!apiKey) {
      it.skip("needs FIRECRAWL_API_KEY to run", () => {});
      return;
    }

    const tool = websearch({ provider: "firecrawl", apiKey });
    it("should fetch a valid URL", async () => {
      const result = await tool.fetchUrl(TEST_URL, ["html"]);
      expect(result).toHaveProperty("html");
      expect(typeof result.html).toBe("string");
    }, 20000); // Increased timeout for external API call
  },
);
```

This maintains your approach of using real API calls for thorough testing while keeping the tests skippable when appropriate.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/websearch.test.ts:15-25
Timestamp: 2025-05-23T01:12:16.998Z
Learning: When testing the web-search package, real API calls are preferred over mocks. Tests should use environment variables like VITEST_WITH_EXTERNAL_API and FIRECRAWL_API_KEY to conditionally run tests that require external services.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-23T01:12:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
path: packages/web-search/src/websearch.test.ts
user: login: shige
body: I will do it in https://github.com/giselles-ai/giselle/issues/935
created_at: 2025-05-23T01:16:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,25 @@
+import { describe, expect, it } from "vitest";
+import { websearch } from "./index";
+
+const TEST_URL = "https://example.com/";
+
+const hasExternalApiEnv = process.env.VITEST_WITH_EXTERNAL_API === "1";
+
+describe("websearch (invalid URL)", () => {
+	const tool = websearch({ provider: "self-made" });
+	it("should throw on invalid URL", async () => {
+		await expect(tool.fetchUrl("not-a-url")).rejects.toThrow();
+	});
+});
+
+(hasExternalApiEnv ? describe : describe.skip)(
+	"websearch self-made (valid URL)",
+	() => {
+		const tool = websearch({ provider: "self-made" });
+		it("should fetch a valid URL", async () => {
+			const result = await tool.fetchUrl(TEST_URL, ["html"]);
+			expect(result).toHaveProperty("html");
+			expect(typeof result.html).toBe("string");
+		});
+	},
+);
path: packages/web-search/src/websearch.test.ts
user: login: coderabbitai[bot]
body: @shige Thanks for the update. I'll note that you'll be implementing the Firecrawl provider tests in issue #935. This is a good approach to track the work separately while keeping the current PR focused.

---

<details>
<summary>🧠 Learnings used</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/websearch.test.ts:15-25
Timestamp: 2025-05-23T01:12:16.998Z
Learning: When testing the web-search package, real API calls are preferred over mocks. Tests should use environment variables like VITEST_WITH_EXTERNAL_API and FIRECRAWL_API_KEY to conditionally run tests that require external services.
```

</details>


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@shige Thanks for the update. I'll note that you'll be implementing the Firecrawl provider tests in issue #935. This is a good approach to track the work separately while keeping the current PR focused.

---

<details>
<summary>🧠 Learnings used</summary>

```
Learnt from: shige
PR: giselles-ai/giselle#920
File: packages/web-search/src/websearch.test.ts:15-25
Timestamp: 2025-05-23T01:12:16.998Z
Learning: When testing the web-search package, real API calls are preferred over mocks. Tests should use environment variables like VITEST_WITH_EXTERNAL_API and FIRECRAWL_API_KEY to conditionally run tests that require external services.
```

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-05-23T01:16:41Z
author_association: CONTRIBUTOR

PR comments:
user: login: changeset-bot[bot]
created_at: 2025-05-22T07:24:05Z
body: ###  ⚠️  No Changeset found

Latest commit: d4a4c145554b9836c1618f9eac4ae57fd9a87395

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. **If these changes should result in a version bump, you need to add a changeset.**



[Click here to learn what changesets are, and how to add one](https://github.com/changesets/changesets/blob/main/docs/adding-a-changeset.md).

[Click here if you're a maintainer who wants to add a changeset to this PR](https://github.com/giselles-ai/giselle/new/web-search-package?filename=.changeset/big-turtles-happen.md&value=---%0A%22%40fake-scope%2Ffake-pkg%22%3A%20patch%0A---%0A%0Afeat(web-search)%3A%20initial%20release%20of%20%40giselle-sdk%2Fweb-search%0A)

<details><summary>💥 An error occurred when fetching the changed packages and changesets in this PR</summary>

```
Some errors occurred when validating the changesets config:
The package "playground" depends on the ignored package "giselle-sdk", but "playground" is not being ignored. Please add "playground" to the `ignore` option.
The package "studio.giselles.ai" depends on the ignored package "giselle-sdk", but "studio.giselles.ai" is not being ignored. Please add "studio.giselles.ai" to the `ignore` option.
The package "@giselle-internal/workflow-designer-ui" depends on the ignored package "giselle-sdk", but "@giselle-internal/workflow-designer-ui" is not being ignored. Please add "@giselle-internal/workflow-designer-ui" to the `ignore` option.
```

</details>

author_association: NONE
user: login: vercel[bot]
created_at: 2025-05-22T07:24:06Z
body: [vc]: #nWuoBm6wtWKYnDILsQVmxl+jzTZD3ejpwkJKx9QdxmU=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJnaXNlbGxlIiwicm9vdERpcmVjdG9yeSI6ImFwcHMvc3R1ZGlvLmdpc2VsbGVzLmFpIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3IwNi1lZGdlL2dpc2VsbGUvMkJ6Q2JZV2Fjakc0cTNSdmRkZFVTR1ZjWUR6YSIsInByZXZpZXdVcmwiOiJnaXNlbGxlLWdpdC13ZWItc2VhcmNoLXBhY2thZ2UtcjA2LWVkZ2UudmVyY2VsLmFwcCIsIm5leHRDb21taXRTdGF0dXMiOiJERVBMT1lFRCIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiJnaXNlbGxlLWdpdC13ZWItc2VhcmNoLXBhY2thZ2UtcjA2LWVkZ2UudmVyY2VsLmFwcCJ9fSx7Im5hbWUiOiJnaXNlbGxlLXBsYXlncm91bmQiLCJyb290RGlyZWN0b3J5IjoiYXBwcy9wbGF5Z3JvdW5kIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3IwNi1lZGdlL2dpc2VsbGUtcGxheWdyb3VuZC9FV0hxTjlrNksyQzNVb1dqQVh5S0JjZm9tMTZTIiwicHJldmlld1VybCI6Imdpc2VsbGUtcGxheWdyb3VuZC1naXQtd2ViLXNlYXJjaC1wYWNrYWdlLXIwNi1lZGdlLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiREVQTE9ZRUQiLCJsaXZlRmVlZGJhY2siOnsicmVzb2x2ZWQiOjAsInVucmVzb2x2ZWQiOjAsInRvdGFsIjowLCJsaW5rIjoiZ2lzZWxsZS1wbGF5Z3JvdW5kLWdpdC13ZWItc2VhcmNoLXBhY2thZ2UtcjA2LWVkZ2UudmVyY2VsLmFwcCJ9fV19
**The latest updates on your projects**. Learn more about [Vercel for Git ↗︎](https://vercel.link/github-learn-more)

| Name | Status | Preview | Comments | Updated (UTC) |
| :--- | :----- | :------ | :------- | :------ |
| **giselle** | ✅ Ready ([Inspect](https://vercel.com/r06-edge/giselle/2BzCbYWacjG4q3RvdddUSGVcYDza)) | [Visit Preview](https://giselle-git-web-search-package-r06-edge.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/giselle-git-web-search-package-r06-edge.vercel.app?via=pr-comment-feedback-link) | May 23, 2025 1:12am |
| **giselle-playground** | ✅ Ready ([Inspect](https://vercel.com/r06-edge/giselle-playground/EWHqN9k6K2C3UoWjAXyKBcfom16S)) | [Visit Preview](https://giselle-playground-git-web-search-package-r06-edge.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/giselle-playground-git-web-search-package-r06-edge.vercel.app?via=pr-comment-feedback-link) | May 23, 2025 1:12am |



author_association: NONE
user: login: coderabbitai[bot]
created_at: 2025-05-22T07:24:08Z
body: <!-- This is an auto-generated comment: summarize by coderabbit.ai -->
<!-- walkthrough_start -->

## Walkthrough

This update introduces a new `@giselle-sdk/web-search` package, including its implementation, tests, documentation, and configuration files. The package provides web scraping capabilities via two providers: Firecrawl (API-based) and a self-made solution (using fetch and Turndown). Supporting files for building, testing, and workspace management are also added or updated.

## Changes

| File(s)                                                                                         | Change Summary                                                                                                                                                                                                                                                                                                                                                   |
|-------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| docs/packages-license.md                                                                        | Updated the version of the "tinyglobby" package from v0.2.10 to v0.2.13.                                                                                                                                                                                                                                                                                        |
| packages/web-search/README.md                                                                   | Added a comprehensive README documenting installation, usage, API, environment variables, development commands, and dependencies for the new `@giselle-sdk/web-search` package.                                                                                                                                                                                 |
| packages/web-search/package.json                                                                | Introduced a new `package.json` defining the package metadata, scripts, exports, dependencies, and development dependencies for `@giselle-sdk/web-search`.                                                                                                                                                                                                      |
| packages/web-search/tsconfig.json<br>packages/web-search/tsup.config.ts<br>packages/web-search/turbo.json | Added TypeScript, build, and task runner configuration files for the new package.                                                                                                                                                                                                                                                                               |
| packages/web-search/src/firecrawl.ts<br>packages/web-search/src/self-made.ts                    | Implemented two web scraping providers: Firecrawl (API-based) and self-made (using fetch and Turndown). Exported provider constants, Zod schemas, result types, and async `scrapeUrl` functions.                                                                                                                         |
| packages/web-search/src/index.ts                                                                | Created a module to re-export both providers, define a union provider schema, and enumerate provider names for type safety.                                                                                                                                                                                              |
| packages/web-search/src/firecrawl.test.ts<br>packages/web-search/src/self-made.test.ts<br>packages/web-search/src/websearch.test.ts | Added Vitest test suites for both providers and the unified websearch module, covering valid and invalid URL scenarios and verifying output structure and error handling.                                                                                                                                              |
| packages/web-search/src/websearch.ts                                                           | Added a unified websearch interface that abstracts over Firecrawl and self-made providers, selecting the appropriate scrape function based on configuration.                                                                                                                                                                                                    |
| pnpm-workspace.yaml                                                                             | Added `@mendable/firecrawl-js` v1.25.1 to the catalog section.                                                                                                                                                                                                                                                                                                  |

## Sequence Diagram(s)

```mermaid
sequenceDiagram
    participant User
    participant WebSearch (index.ts)
    participant FirecrawlProvider
    participant SelfMadeProvider

    User->>WebSearch: Call firecrawlScrapeUrl(url, formats)
    WebSearch->>FirecrawlProvider: scrapeUrl(url, apiKey, formats)
    FirecrawlProvider->>FirecrawlProvider: Validate URL & formats
    FirecrawlProvider->>FirecrawlProvider: Call Firecrawl API with timeout
    FirecrawlProvider-->>WebSearch: Return {url, title, html, markdown}
    WebSearch-->>User: Return result

    User->>WebSearch: Call selfMadeScrapeUrl(url, formats)
    WebSearch->>SelfMadeProvider: scrapeUrl(url, formats)
    SelfMadeProvider->>SelfMadeProvider: Validate URL & formats
    SelfMadeProvider->>SelfMadeProvider: Fetch HTML, convert to Markdown
    SelfMadeProvider-->>WebSearch: Return {url, title, html, markdown}
    WebSearch-->>User: Return result
```

## Poem

> 🐇  
> In burrows deep, I code and scrape,  
> Two web searchers now take shape.  
> Firecrawl swift, or self-made might,  
> Fetching pages day and night.  
> With tests and docs, the path is clear—  
> New knowledge hops in, far and near!  
>

<!-- walkthrough_end -->

<!-- announcements_start -->

> [!NOTE]
> <details>
> <summary>⚡️ AI Code Reviews for VS Code, Cursor, Windsurf</summary>
> 
> CodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback.
> Learn more [here](http://coderabbit.ai/ide).
> 
> </details>

<!-- announcements_end -->
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpETZWaCrKNxU3bABsvkCiQBHbGlcFAxcCkVsBml0SHIAdx40BgBrNFJ4tDZ6AAMAASIkEh8SMERaVIB6BJIBcpJnBlhcyAT1BCxcWBJIAHFi0shmfAx8f258DRge5LSM3t58CXglZAAzbAwxeFG0L3V5dfH7BipueAwidAx6EgAPCJTxK7a65MymcPYwyAF8bqQAAS0AAsgAZG70UHOVK0fAJLDHCjMaiIaYASVClwYXmwa0guAS+B4kRWSj48GY3C8JDY4WouwwiC4ADF4P4zmgEl4ADRtBDNMI0IhUGjIdqA7q9Ar02gqWlVdYc0RUHlgISIVoAQSsGP5mHoAGUSusYUoDYxsIgCMwUNTafTcIzRpBrZdrhhGVJIOsSLghYaYNgKLcEUiTiCIWACGAYRQ4eGFBgpBREEzpnBerqMShkLhZNwyog0H7+VSaXT2HQ3enXtBCyQjWd4NxQkHpZAAF6KSAHARUFwaNyzbgpdKfFixSWwOJMan+HrMyS9cWhBzqXoCbDwLyhGeQABqm5tBtu9mw3EmFFwyDIK0iGCdkAkzngCt6X2VRBDLojfBzSBUhIWR0UgAAhHcvHoIMlCkLx8G4Z9EBbNtkGcXp9kQEkcTxJRaGmbU91gfBsCIWdOzHeZMlQMZQgYEN/HCLx5EuWsSHLbFkEuGhbhrAg/l6PBd3gLsazYtALwERBAmCcIeG8Xx/CCEJfROCtHXYD1IAAdTqKwFkQAA5RRek2bZxD2A4C2HAxoBCbTcPxWItnUQkQkQfkD3ZTk1V8QC12QZSdwobTO1yVkMRsABRABhGxtR08EAH0c2SgBpaKAE1WiDXJDwxaBoqNaBkp0wqgWS6KAA0ipsIztRSnMAF4AEZWnvDlRmfV9Qo/fMSQoLZMwQZBbHsSgpHQ9DSUoWSkE3NS+E7S9520vSBAM0hjNM+1KydP9bIsSBYpYJ1kAcJwXBHXpRhYhRWB+V1ug8RS/Fk1TUHWSI7U7NA8BIih+WtShIFyRAEFIXIvJI96VJtGtJP8FYSCSb6WDB/ICFkbI3wwWRRlaOR7GdF4iC4AAiKxaTQGT3pRpIXuQXgUHWSACewSBYDQH1xDYEAKcgEz+GlPh53O/k/ToARx35E5aCQBjEHTUZ0P8WaZPktjKLJVYa3lZ1bP0YxwCgMh6HwNn/tjUhyDFGtxfYLgWcEEQdimv55CYCkVDUTRtF0MBDBMKB3G49CsGtwhbeUGh6Ed8IuDVC8rvkYnveUVR1C0HRjZN0wDHhBhECqKiJ2kMADhiZkSA0ZhaA4AwKebgxju1DFiDIWOa0u1EXH4Nnmkwbabv4DB7qHq5em12ZlVpPMbjdbgDdXElO1TFWsAwRwBFBy3CVHccFkgCmXlkIgENUWRBfRu0JAABg0AAmDRWvvwkSQf5/X4AZmmYWAQ9DFqMXiuB+RVzIHTS4yJUSWQwHLPgY5ujIG5j6XeZBhiKHgMqOgtlzCWCIjQMUTIBoH16EoXEzg/zIH3g8a8cd+CIOwAIKukAtLiGkEYEy5Am4twMBAMARgy6GRqHUBoTRYBVBitqAAIqCaKdcG68Ipq3AhHcY723oL3Zw8h96TxHgYbU8RUaQGkXI6KvpdxmROOFQogxaTlEqKI+oMkJGtAbEWZsoU2x9ngAOHRXNaaCQwWgWg+ERqoH7IOeQSxySxHMjsKyhwP6nHOJhc8DwnhiDeAID4n4QE/DYlGSEQZ4yJkREtWBYEsQXivOMW8hJiSkmWHrNMbIVRch5DDeAQoQb5lmLKc2H4lQdL8hqLUkBAJBmCiqCOky9RARAmeGCkATReDNKExYusKQH2oOxZAXpxA+j9AGWcHZZjQBDGGCpUSAnImBGCcEMZCBlPhBUr4G8mRtA6KRUIMzQqvEwPM3MwFZAjV6GY+RkAlDOl3NxZkzofB/jCDaIaiTmTAxLJkB42RKwbBOP8QEsTWmeShGQ4F71ECTBriTNFuAQwcQFD02cTkCSdgAKo2HBPycQuBaT8mKcsyArykxfFAZidsXhsL2CLAwbB8BYidUfD1N8/VjF0BrPczsPlVTcl8MSnZHYHJXDlngA45BkBwRKIhZ84tDT4r4NuXcCsTWEkbIwHoaQPSS3GLAsmPLjVECFbiRoGBvVkrWC2XeKCEQpKGl0Dy3zAQnBnL88lOrOn+T1OC6F+AGIHTgWw6k3N0xiXzO6ksJz5DuleOFHstBWi3P7kGA1sRzWpFIf4Wkr55JKCLHxbYCr0RGHwZMvcscSEpM7BQrwVC4E0LZnQ+pNYTieBYT0th4R1BDqMFAbhvQAAU07sFszuvIUJtAZqQosUXRw7BkVz3SfQeEsQ6KYIVusc9+MFBKEYWw+49CazsKHQASmHM3FR/CC7CO2s48RFBmilyPqQDQmpRiNwg6oyZ6iu6aJTn3XRg9uZT0QEYIxiQwYwdrmhjArRH0Lx4pEWg0RNU2IGXYmSpRHHVFqC4xoCGWhzHLhK6FJBlQWvJVR4Y/pQnUEkqy7SXo2D8k+aMIVvBJDUF6Daag1oc3sH7pMHiDqTpnVGAAKSNJAA9uRUSXFyCB/kxV33eEPbZqItIHNCo8U2VCoQCxFmQDZgL0gHPoHVlS0Q8q6AGh8DwfAPEwprwGQrG0rQFacgIP3X5nhbw5qkyhbxDTWX5KcLcEzTroLaWCw4bgXm3VFg9aIVI1Xci3gYIHMY0VmDqHqyGzA2lKswRvNg54Edn0kH7UoQd0gfUomoGTH9vQDy5DUFOaGZL41htrYmlbKw1y5BzbkJdN4Jk4Ogv+mkPT1D3VRNwfpixkO9EiACMl6hkC5Co6h7CtGloufnv4bCeI4H8lq/QwbgC9r1M2/NCLBZaQSg6OgK8kQNNaYa+Q0Qs7iGq2mDIkg8EbU/D7ebMgcrYglZgI2LxrZ6KjG/L+edPL3VKHE9u1Wf2TK/qDJc0Mbz4HkqGx/fAvg2u1Y2+c1cqktug1yPtkIh3TFbD5pjqbZOh1hFxM5TGcphnKl8nq8ZG22tXP5xL88uR62K6Ir4EnA7yfIBBm0cYHaqK3TFnJhC1x/B+iYjEUllwFYMEZICzBYwJgknUt8UM+whPH38EUVF8gZL0u4HgrDhCJ0c4EtOrHc7J20IA8ui2TD10deA5w/hQsTFSZnfn9DmNE8lAcRUHjYjXECdaAeVTWBciP3760Nin2nsl14/BxDX2aOD+2HhbShXaekpO7efkcF8dq5m6S2Ck3Scb/A3wgRQiR9wY74hxACGRkG55BoNc1+WTKIzzhu2WmtGOAIwPD1w9K9h2MYzVSG4aB/bhSFZoBFjsoUBeB0ZbDoq+g/Tkq5D666o8itAjDMbzyoCMZRAxD0A1rXCdjHhrjuQ2jaTfTZCowu4iYlb5iJr3KEosopj7CrBkqvgHD0CcqQiXC5ZgRZhhDMGMFsGEGhCpjRb9J7KdgJKFrdCRAJBzKUCRB8AJBLiQBFBSCRzDD7AwI1j8GooejTCsgnC8GsFcqkqBThb5K3Ds5eg+DyDxpjz3TYLkpKrdQ/C9TvgsIygFRFQlRlQVRVS1TRT1SNSpR6iD4XT+gpIUytQUz8iQIhjaSZKUCWEUrB4+BqxmERAi60gEQzDSBS42jIDNDNa7L+azxQGFr+D0qhh3gAaiAMJLBFgjaxC5CwC4DMAQFkq2awhm6/CdiA7eChCuw1F44npzTyQoRkBvj4CpEKCph0BJwfREGvBnpqEJj85yzjzyDNGtEIJ/CAI5omG0wyQ3iSaA5k5PoY7v7Sh0x1GUAcKb7nhCGfpFECgYJLGjDWLzZcRwzBAIy0A8qzCAKgyoDlFXKIwyHUgFi0o6GQDRQpCzgGGQD8EEElqLzxF8T0Aq5pr7xPzvwyRfCXopIpDiyKDo6TCgLvhWH2AIRJDxEx5KTzF5YjoZ7jo47MhTqzB14skLqXbF5/prqsIV6kZV7Cz6KxACRL4rql78lbq3EADcB8qAH6VsYSM0FGBB9G8Boyeq1+IQt+XePyeA762Csg2kmAMSzCrCgE6YRAhyDKw6mGUGgiBgVGo+7e/Gp+5+CBmat+GGLcbcj+3cL+qc7+Ipgp5GJiKBrmYMnpfkupDG4QTGLG6JJIraOSqSIBxBpRTIDBEJOB6ampPIE0FAKwMQImS+M0XwOm8krafASmZxkkAAWr2ChD0KiIAaONsnvMIIMdkf9pjuJrEPsFST3FyI1jAmiOgMgBTH3OUhgILEGBTJsV4HOeeEwBQIDtSi6tcD5jTj4iFiZpce8VUmSp2MAY1r0XuN0kKJQQiVyjyjdoyoKmStOfzsmOKqPKub0OKTBIgLINsLAI+KRBsJma6ODCOSQGAW0c6MBDNGwWeBSqCkKkCohHArHgcDaO/mOQ0geqzv9HuEljsYCIOQiKxh8YgGBpADUgYVpg9jeZCJhX8RgmcI0OKHEBmn5IwAcIUgipgDEEmhRO2S0vhPBSBDmuIV8vUTAjNKec9vSSkmxXqnxXEG/A0HiYSFSCQGmhpFWN8PQCsJJLkFYD9MUBoFQDEIrhiGzD0bJSrjQgaWxjJVSqrNPOYcHixesNoHiP4JxAfFITIWuSmkjkCiAXUX1AAWwMrAsNMAAPJYAOAMD+4B5/L+hXJzIDHZJiraDba4GzDSWGHcpqV8qHq3zSbOgGxoCOYPIQhCrPmioFLhAr5ia4WLZingnJ4RAejcRsw9bKw6EjioARnzxk4gEOCzosWyEnDEa0DmrXD3KXDwlsGkrBSqQYl4B3HYHMjRAJWbC+DSWUrUoySkoGomkY6/HNLdVlAywyT0A8SUDuW8X3K8ZpkXCvB6WQDyVX6MltzMnUJsmY6UKcnv5fm8nmkboCm7qTJhIOyqzOjyRL5cAamX5eCGWCWUBGSkGUzRl6oUzT6UZH5j4n6SJn4MAX6IFeC6lV7aiQ30CNlaIFGtlw1gzvVeAbQmgSLI1xIUA43D7UTSDH5umE0en5lk23i5AU1U1U6eJ+bnEM0RRC0s383s2tLT6+7+D0DFU02nAtnya97Ol80SJVBE0k1eki1i1CUVkw2hAy2NTghRU6TRQyLJRGjxS6jRTJSshRU2CgjajQBGi5Qfb+ChJLEADaU5nR4YURJ8i5FMAAulzbrfjfzQbYLYjeTVAJTUJduVLSFtyTePDTbsRbQF4iASQHoaRX7W6GGq6PvERbUGrb6uOUPvHa6frYbZjVfibWneLZnUVtLUXrnYzULUXUWDYNIH0XqYCOdvibkCGBAfyG1vecboucbjVYiHHXjc3QJkncTW3cLVqKbaCb+R1mJa6DLdJRBQejPVwNoa6umelCBFfe1a6phYgAAPxcD5211D0l3123hB3R0gZcDI3nXABM1f0j0jW4B6Br080ul8Yt3J2k3k32kH5Onr1wOb2G2B4PDen35+mdxP4MLaLZZEaf6hk/69m40wN60YPn5YP3CxnoHxmYH8Tcx07Mgi6rDUUw5gBlm+jAWWHJJdrP4pKPUE3NIc3VLYjUj1L5hNKiP83iMkpz1M3y1s0dmc3tFrIbJKCqMCaK0UgS4ISvAHjSgch7U1EriKM7K1nJiVl5Y9kDUyU8N90NJAFgUQWQEWSTrFW0FWOUCO58R8CpYvDZIMG0yxA2Y71f0ePtGcbaO+ZpIePkU1Ks6XADmQAa3Nl0j6W6PND6OUDNha3l2SSuRV2WWzBEg4TSM3g1jVma3ZOL5ejrovUMEGxfK0IdCgx1MhbTCnRrnSAbkegsSWjd207nFhMfa5OwD5MaMKmUArhq2wFMz1OoiERhIWGDmyBwVkCOArM5N1Cs16PqMTKoBMXCO5ka1inbxsD2zHEo01mkGkp3UiSkxhSNjlClhCRYq3Rsx1N6xbo4JphHQELfXzq/Wib/U/WF6AYl4KRl6bq8o7r713AuPw1RPuPgGtAHoTOaqwGgWJOYu/CHZG1+QObIs52cBgxxPmgJPF0xPYsHDhOLMYz4t0uEtD4aAG2mhgCohKBkud1CVA3m2YCUu5BTMzOFPZNYsa2lOxV02ST7yy2I3ivqO5QW5aM0sqv3P8sQ2CsuP8TuoTPw1asc1YvQKyG4sssmutKSuog6vp1Ab6vxzQ0ivGsHMK3HPSu9g7N2iKs70zNo1sBqt5DUubIBukFkvIPQZoPj4C3E1xM8ubLak2g4OYZ4MaLCNEOEYf4kZkbkMEH/7vFwFn2EvH1YAzwygJu8skDIEebLZBIXo1i5l4Enj+aBowGkHEgJh7GJpMCpgEV0HwlBjzVcphCcG6FR4jvgjGEVOqQPEa7dB7KsugGEuSEIh+VyETt8BTtBa5kLm4C4D3YcBVBVA4oOi1zzhVAUwVWSEkDabVwTFTFrizFfHttLHFL8B4C5ZrG3Zh0VI5Z4BCq+MftBgr1YAAf2MwlCgEEHE3EiHFEyUVHkAWxdnpUgKZUyCQ5NEtFtF5RgetDXENFgQ6SKHvuPILxLU/EMVgxL3NL1EQkQykQXa7xxBjAYA8OtWQmAoW74f2AkTeD0AsdAp0hthtUApEDTAkcvHrHLEzkUf0kxZwG8eEcMf8fMeYTxCjAceidcfXB5S0eMcCeCQomcfX0Se6SKG+MYSvtUcDt0c3Ea6GfqeafscicMeP1EBcGzt5GboODqxiH8MADktlh7BpqK0QFRZxYqPw2dq5mWFqMgTLY85KlHDCz9/8/oXbqQYAdu02bbPnNheJGzFJl1K6XQAyHhxUpU5U0AlUNUdUDUTUwRm6D4Th8kLhH46eX1RCP1Oe7JeeAN0LPJq6IN5e0pSLe6JIIZKSQNI38LApcpyzipth56yprHJiapViilBBweB1bZn59wSAi2y74Fpb/De+kGKDTd6D7p8b3L1bqbvpai+DAZ+GASeixGBiYZSQjjVL93SbItcZ6RzGWBDVbOoeVbmyqZYjdTB4kkyo9wtT6jWQbAtjFtZKkkcXAzowm5GTTZ8rf2VFcCpZLj5agW70vAORW6taAlHNx5OVYFlKfRl5dBWuBIsFBV/KlVkI0X9VZKIq7ydVmgDjdb+ycBSHXRTa8gAkHyNx3PKSAvWAB4kWcqyoR9P9i2yFk6B63t1UXMjQm5Tzpxzrv6LCeaHaiFCMoUiAqQfwikYR05/j5F3BH5fDXjSSEJqAQVP5f5AF1oVLGLbRZbTxRPopswHBBpHPJyBRNFH7vPoQxVHKt5/6WSrjh8mQvK88uZSMJApA9wwaOPxXE8owqYafvQH7AkivfwSXz0swqXJFVSImwJlRi8aVbDsKWV5KHPmSplDCmfjKkuNnDC8flS45B6IHtwVQJwivzvJR7v5bM+zkd4/lfAk101f2c1rTtF42voHlmq/oQoqXw6I63XWerJfXf12OULi6TrwN8343leUAp03F8kjbDcf36yNL4bbAlMkPSg2NRLa7rGy3pct1kibJQKnWhK39MmBPN/vDQ1abJrWBjQATGwJogC/+tcDulAJhbjMqa8A00Jq3dZqNtWYQFWpa1+izAYBWtFAVQwTrwM7uYAh7lgOii39s6cAsGAgKUBgNR6e4cerOEnofZL6unY3P3wfridF6OHMQR6GXp/sMAUgq4NA3LiwNgBhtDAZAJYE4DaYh9N3tAXYEndz6QgsznNiqRv1rM+7VooLAAA+J8MDtez/ogZIAzUPQNYCMoyRgAnA2lsPR4GQNFBIiOgTQwYHrBwBmAvelG0dJAC0B7WenPAAk40YfSKidNrhkzav53uJDXNoYnIajMfEX4GIYzi+TqkohGAb8N9kJiBIZAd7SOOLQv6UMlB1DZoOlg6RZYwUMwU5tEJ/AskU+5sGaKV2TAM4OhCEVyojFCCHZOWGgTlgQBFwlxCh34KoGMAgFT4RMKvYQg9AuC0g+AWvbPJHi/YGl5wqwzVFYnhQCR9KwTBoZlnGBfotE/oGii9nXCkQEM5CRoecJSTgwEMiuF3m0LyGuhryg5CWgkzGaPoAmOyIAq8OhSPDm0GSe4GzzD4yg5hJAZKI4wmR5QThoIs4aFHiSwFdhu4Q6J9RBY9cwW1QjktfwpYSk4WUpRFpXn3QXdd00bWgRvUQy3hLwGgHIRJ1vDxCH8L3PDFm2DKfdK833XobkI6H0YtUAyBkXVjt63A1hQPBMlgW6IDIIh/NU4TUXOERJ8iHwjoUsJwQ0UDMMSBLK/w+yYM+I9DEWtESaYI4lCSQwtPvCyGhBCRhaAEdZlyC0BWRhIIaDWwqpBgIOmHQEHFC9o7lQgv3GzNIGYBhYgwp0VgBZisw2YGAmoMLOl1My/oqU1kRbAqSQAfh6ANmRMeoDJhcB3KUqN0aDjuG8U7sUxcYm4XTHgxCx2TbgFwAiDBB6sg/CDiiKVH9xTmNMZDoJGRCflYSdvZ1A6P6xyCXRdY2fmYT6HIpUAQNIJLnncp9EKWtYbSI6LExpNn+34OjEsxFHi448KGHEWOjxGToCRA3IkbN0lKg0H+gpAVvrEaozil87+THmqORRt8Xw74MGKk3IDLiYhvg2DP4PpG1YmRbQ2MgeDGCkhdgfAB4Ed20i2iSEVIh0oflpE3dJEFRf4CULkG4NnuGbQhikOIY5svu5DSzFFSMj8j2hD6LbibgoCISp8ZQ4JJUKEpbAgRco1AQqKbFiBlRLQjYFt1RL4lJIPQ5kZ8IX5B5hGxMXIBTHYmIAKYXAEOiexjrmUbRi4iTJU0JC0wO08NIbMGzBgHYjsyk+STbxlRRZNRTxd7J+1C4NJrOAw4RhJG/SOijuiopif3BnAVtNxbmCmAACpHJVQYJlUGcnY1hxqkhXJpNt5zN/qNYbcO3ww6ad0A6zFCr4G4kdCTgnorrriLP6kJc8kLMFkNxqaws+SJ48kWeKFhvFrMFGKKURPnhv9+QgEo8aSI3QZwEW26WIByToAgZ780Ep0hgCQhgAsuVKFILXBxitE2RiQghj3AwnZsQy75HkWFI4nkMpM2o8FrkFcr7B8ARAVoLiUtHlMZQ3AZqcwFaku52pJZLqUHysTTAjQsqeVMkWGaSYnsYMCmPkF1xuESWhuTUAAO7z+MvkuQVqM/AACsr8EIsKGlGBTMuFQhwodwWLXADMGuQSVsBtDjAFgokyAC9NagAAWDQLDJUmCTtAUM+GX/AACciuABKLENJq9euyWR7DzV8ToV9Cj0jnNZwe7bjM8ANfcclILw38cBc3MkdVMFKUiGpV3eifQOcQE1k2mgVkShOwwcjkhQZD7qQz6roRwpXyDAiD3SYFsdwABYUTKF4wE1a2qBL5mFFmD4Flq7bEgmwCy4iYXxopJpHZgTToVRQpEI9mPCLZr9HI9BFgtvwx50QgEhPLfgtWeJYBm2swCmBgMFjVlRKHIdCgQXNmXgXw8zXSYu3oiDl5x0fWADEzh4L8p2unHyuuxRIr8Jyl2Gorgh7K4kceAhJQpEEvAs8/AYM5MArAin3RSuFscrp+RTBdQnwzhVVG4TBiVcvCNXOrn4QCJNcMQG2EwhHN9AH9OgenF8K7NHbN9WSOfCBq+Uyomle8tHFTtm2zpmcVRE0VPL51tLV90wHWInu0zZj8EbZ3HegOuScq0oIuDKfbmDGVkMSJhZNKmaCz3EEyIWV/FKQzOG7HixuWU8GsKR5GkIypGU9+SzMW6jRQR6wU9DJwvQqkNuqkdUpfIkS8zdSUEjmbBJUHn4YFAmR7gkNQkWj+pIstIdhIoy/dcgqC5oAw0OHfSkyfjOIK5BwTXVo8d1Ito9RH6nliCsBRUiMVCDVlJGUOU7E9QzLz8d+dTWshsCWZNIZIY4W5giLRZC0kZGAt4bMF+6GyyeA5c8DdQoB0LSEUU1cLMHkYSJhcfITXLPleC5AP6dAUulUjowxTthCfH+kFnMFLk/0odFYuHXdHng5JBUwtCorUV/ZGgQoOpjZg2gE03xRAJmkjP8X81AlHghzOCjpjWd5wagdsTxBJAlNK6XQd1GKzqABK2hh2UeGkoECHNmg4DMeucUjShRo05KcLmIDPn7wTyYFZ9HJn5AZVLgGs3oBz374F9vgWsLAMB3I6gdZBo/extwRyUE1oA+AEXNPiIR0LtJqvDXPpRjkxM2A3QXsH3MJKTYjJtFMlBsMSLP1+QY846ksHOqUoRcKwWtIkrBhTMClvA2yO8PVhA1g+hC9JVfLQDQU4ggysJZkphxIcZoLyiRMMtGUooYaJZCiuuBKA1EaKwVVHKFXvbnBeF0BSuclx1j3NJl0WGheSmZFbs4CO9Ajuo28rHSPs6LAlm0RMazAOeSFNsFmSzQgoQI1mHCn0XwrCdTOHnVmJp3YUdlaAzi9EnXwU511SKaKoAv9z5Z+MgYKASOSkQ/7xNomhLQlc0tHZBg4x3+FQKimeBfIa6+RTiuEFIQxzUyI/Khb6nugJ5nAU1aQFyRWiBMWI2kP2bfN3GbDyUEEjnKlIYRMzMpLM8Gg63RKpLjFtAUxWiGKYV0d5Sc6yMoClTnSo6kAawQ4pnIADG6nMgIdzP5rwLyWHijqacvuUSJAlwSxSipxBnVksW2dP1kLW/5uj1lpKqyGDFvogQsWZnBwZGqQVoDDaRC2AHGvPE0Lxliar5QJnCUEDNkfA+zjeHkCfZVW1mHNWzHBgdqlA+asLFWtqFfi42Ma2BSLWHCNqfVz0VJaEpTVvKgkra5oKmqFohrdIyattW0IiU0DJ1dI6dXWobW6sDWjWHJXktgDnLcAXajNY0RnrlqPOFVeeoVRfXic31tHA9BWqFQdFHFq9azBWqPV+CT1IAs9XOvjW0KW1K6gTD8oJVI55lJEPILMvAIX1wC8goNH0tfrv0fABdT1b/X/qANXBJAYAGcu8FQNQNn48DbWr3XEKoNC6m5fwwvn0bYAB6ZkVwDg2bq2hADXddJH5oIaPxvNKdRBrY1xqo2hgAwCHE3QWwrYeAaOFgudaPRE4fgbkG937jpxTIVALOP7FzhBxpNpsB6D1lwDJRVgiAZKMjAVS11koOmI4nnBk33wSAP8AAOwMAn4Lm++OjPRkAAOAAGxoAfNAgHzQwFhk/x74oW9GX6D820BXpfmkBffB/hPwGA6wVqAwD80/xA4wcIzSQCfgCBYZUWhgK9J80/xXpSgVqK9PRmvSGAP8XeK9J/jrBaADAdGT/FhmqBYtT8V6QVthlOasthmiAJAD813sMtaAH+GgD82wyGAtAPzZVrqChb74oSBgGlrG0ub1gU22GS5toACAf4tATzZJGNgybaAsMtAJNrhmvTztbW3zT/D81La/NrUHzesHRneLjtJAV6attoDozAtLmn+FVr639aoA4sdQGZsvSWaCc1mugMlHNhZagAA= -->

<!-- internal state end -->
<!-- finishing_touch_checkbox_start -->

<details open="true">
<summary>✨ Finishing Touches</summary>

- [ ] <!-- {"checkboxId": "7962f53c-55bc-4827-bfbf-6a18da830691"} --> 📝 Generate Docstrings

</details>

<!-- finishing_touch_checkbox_end -->
<!-- tips_start -->

---



<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=giselles-ai/giselle&utm_content=920):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Explain this complex logic.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai explain this code block.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and explain its main purpose.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

### Support

Need help? Create a ticket on our [support page](https://www.coderabbit.ai/contact-us/support) for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai generate sequence diagram` to generate a sequence diagram of the changes in this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->
author_association: CONTRIBUTOR
user: login: shige
created_at: 2025-05-22T08:44:40Z
body: @toyamarinyon Please review this pr if you have time!
author_association: MEMBER
user: login: shige
created_at: 2025-05-23T01:18:48Z
body: Thank you for your review and improvements!
author_association: MEMBER