Repository: cloudflare/workers-sdk
Pull request #8492: Use a shared Vectorize index for the Vectorize E2E tests
Author: login: penalosa
Labels: name: e2e
description: Run wrangler + vite-plugin e2e tests on a PR
name: no-changeset-required
Created: 2025-03-14T13:28:43Z | Merged: 2025-03-14T14:35:06Z

Wrangler has E2E tests that ensure all resources can be accessed from both a local and remote dev session. Typically, these perform some sort of round trip operation on a binding to validate that it's both available and working as expected. For instance, for KV this might include writing a key and then reading it.

Implicitly, this means we're assuming a write operation followed by a read operation will reflect the data from the first wrte operation. This holds up for all bindings (even in practice KV) _except Vectorize_, which is [intentionally asynchronous](https://developers.cloudflare.com/vectorize/reference/client-api/#insert-vectors) here. As such, our Vectorize E2E test waited for the results of the write operation to be available before trying to read. This is _really slow_, and means that the Vectorize E2E tests often timeout (past 90s).

This PR changes that model for Vectorize. Instead, we write and read from a shared resource across all E2E test runs, which means that the read operations are reading values from a prior E2E test run's write. This is slightly less valuable as a test, but for our purposes it's perfect—we don't really care about the semantics of the Vectorize service, we just care that Wrangler has hooked up all the plumbing correctly. Any vaguely reasonable response from Vectorize validates that.

---

<!--
Please don't delete the checkboxes <3
The following selections do not need to be completed if this PR only contains changes to .md files
-->

- Tests
  - [ ] TODO (before merge)
  - [x] Tests included
  - [ ] Tests not necessary because:
- Wrangler E2E Tests CI Job required? (Use "e2e" label or ask maintainer to run separately)
  - [ ] I don't know
  - [x] Required
  - [ ] Not required because:
- Public documentation
  - [ ] TODO (before merge)
  - [ ] Cloudflare docs PR(s): <!--e.g. <https://github.com/cloudflare/cloudflare-docs/pull/>...-->
  - [x] Documentation not necessary because: internal testing change

<!--
Have you read our [Contributing guide](https://github.com/cloudflare/workers-sdk/blob/main/CONTRIBUTING.md)?
In particular, for non-trivial changes, please always engage on the issue or create a discussion or feature request issue first before writing your code.
-->



Diff:
diff --git a/packages/wrangler/e2e/dev-with-resources.test.ts b/packages/wrangler/e2e/dev-with-resources.test.ts
index eff2f65650..06dcb3e3a3 100644
--- a/packages/wrangler/e2e/dev-with-resources.test.ts
+++ b/packages/wrangler/e2e/dev-with-resources.test.ts
@@ -518,8 +518,14 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 		}
 	});
 
+	// Refer to https://github.com/cloudflare/workers-sdk/pull/8492 for full context on why this test does different things to the others.
+	// In particular, it uses a shared resource across test runs
 	it("exposes Vectorize bindings", async () => {
-		const name = await helper.vectorize(32, "euclidean");
+		const name = await helper.vectorize(
+			32,
+			"euclidean",
+			"well-known-vectorize"
+		);
 
 		await helper.seed({
 			"wrangler.toml": dedent`
@@ -530,39 +536,37 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 				binding = "VECTORIZE"
 				index_name = "${name}"
 				`,
-			"src/index.ts": dedent`
+			"src/index.ts": dedent/*javascript*/ `
 				export interface Env {
 					VECTORIZE: Vectorize;
 				}
 
-				async function waitForMutation(env: Env, mutationId: string) {
-					while((await env.VECTORIZE.describe()).processedUpToMutation != mutationId) {
-						await new Promise(resolve => setTimeout(resolve, 2000));
-					}
-				}
-
 				export default {
 					async fetch(request: Request, env: Env, ctx: any) {
-						await env.VECTORIZE.insert([{"id":"a44706aa-a366-48bc-8cc1-3feffd87d548","values":[0.2321,0.8121,0.6315,0.6151,0.4121,0.1512,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"Peter Piper picked a peck of pickled peppers"}}]);
-						await env.VECTORIZE.insert([{"id":"b0daca4a-ffd8-4865-926b-e24800af2a2d","values":[0.2331,1.0125,0.6131,0.9421,0.9661,0.8121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"She sells seashells by the sea"}}]);
-						await waitForMutation(env, (await env.VECTORIZE.upsert([{"id":"b0daca4a-ffd8-4865-926b-e24800af2a2d","values":[0.2331,1.0125,0.6131,0.9421,0.9661,0.8121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"She sells seashells by the seashore"}}])).mutationId);
-
-						let response = "";
-						response += JSON.stringify(await env.VECTORIZE.getByIds(["a44706aa-a366-48bc-8cc1-3feffd87d548"]));
-
-						const queryVector: Array<number> = [
-							0.13, 0.25, 0.44, 0.53, 0.62, 0.41, 0.59, 0.68, 0.29, 0.82, 0.37, 0.5,
-							0.74, 0.46, 0.57, 0.64, 0.28, 0.61, 0.73, 0.35, 0.78, 0.58, 0.42, 0.32,
-							0.77, 0.65, 0.49, 0.54, 0.31, 0.29, 0.71, 0.57,
-						]; // vector of dimension 32
-						const matches = await env.VECTORIZE.query(queryVector, {
-							topK: 3,
-							returnValues: true,
-							returnMetadata: "all",
-						});
-						response += " " + matches.count;
+						const url = new URL(request.url)
+						if(url.pathname === "/insert") {
+							await env.VECTORIZE.insert([{"id":"a44706aa-a366-48bc-8cc1-3feffd87d548","values":[0.2321,0.8121,0.6315,0.6151,0.4121,0.1512,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"Peter Piper picked a peck of pickled peppers"}}]);
+							await env.VECTORIZE.insert([{"id":"b0daca4a-ffd8-4865-926b-e24800af2a2d","values":[0.2331,1.0125,0.6131,0.9421,0.9661,0.8121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"She sells seashells by the sea"}}]);
+							await env.VECTORIZE.upsert([{"id":"b0daca4a-ffd8-4865-926b-e24800af2a2d","values":[0.2331,1.0125,0.6131,0.9421,0.9661,0.8121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"metadata":{"text":"She sells seashells by the seashore"}}]);
+							return new Response("inserted")
+						}
+						if(url.pathname === "/query") {
+							let response = "";
+							response += JSON.stringify(await env.VECTORIZE.getByIds(["a44706aa-a366-48bc-8cc1-3feffd87d548"]));
+
+							const queryVector: Array<number> = [
+								0.13, 0.25, 0.44, 0.53, 0.62, 0.41, 0.59, 0.68, 0.29, 0.82, 0.37, 0.5,
+								0.74, 0.46, 0.57, 0.64, 0.28, 0.61, 0.73, 0.35, 0.78, 0.58, 0.42, 0.32,
+								0.77, 0.65, 0.49, 0.54, 0.31, 0.29, 0.71, 0.57,
+							]; // vector of dimension 32
+							const matches = await env.VECTORIZE.query(queryVector, {
+								topK: 3,
+								returnValues: true,
+								returnMetadata: "all",
+							});
 
-						return new Response(response);
+							return new Response(response);
+						}
 					}
 				}
 				`,
@@ -572,10 +576,11 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 			`wrangler dev ${flags} --port ${port} --inspector-port ${inspectorPort} --experimental-vectorize-bind-to-prod`
 		);
 		const { url } = await worker.waitForReady();
-		const res = await fetch(url);
+		await fetch(`${url}/insert`);
+		const res = await fetch(`${url}/query`);
 
 		await expect(res.text()).resolves.toBe(
-			`[{"id":"a44706aa-a366-48bc-8cc1-3feffd87d548","namespace":null,"metadata":{"text":"Peter Piper picked a peck of pickled peppers"},"values":[0.2321,0.8121,0.6315,0.6151,0.4121,0.1512,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}] 2`
+			`[{"id":"a44706aa-a366-48bc-8cc1-3feffd87d548","namespace":null,"metadata":{"text":"Peter Piper picked a peck of pickled peppers"},"values":[0.2321,0.8121,0.6315,0.6151,0.4121,0.1512,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}]`
 		);
 	});
 
diff --git a/packages/wrangler/e2e/helpers/e2e-wrangler-test.ts b/packages/wrangler/e2e/helpers/e2e-wrangler-test.ts
index dd0f80cbd7..cb2333bd29 100644
--- a/packages/wrangler/e2e/helpers/e2e-wrangler-test.ts
+++ b/packages/wrangler/e2e/helpers/e2e-wrangler-test.ts
@@ -112,14 +112,18 @@ export class WranglerE2ETestHelper {
 		return { id, name };
 	}
 
-	async vectorize(dimensions: number, metric: string) {
+	async vectorize(dimensions: number, metric: string, resourceName?: string) {
 		// vectorize does not have a local dev mode yet, so we don't yet support the isLocal flag here
-		const name = generateResourceName("vectorize");
-		await this.run(
-			`wrangler vectorize create ${name} --dimensions ${dimensions} --metric ${metric}`
-		);
+		const name = resourceName ?? generateResourceName("vectorize");
+		if (!resourceName) {
+			await this.run(
+				`wrangler vectorize create ${name} --dimensions ${dimensions} --metric ${metric}`
+			);
+		}
 		onTestFinished(async () => {
-			await this.run(`wrangler vectorize delete ${name}`);
+			if (!resourceName) {
+				await this.run(`wrangler vectorize delete ${name}`);
+			}
 		});
 
 		return name;


Reviews:
user: login: emily-shen
body: .
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:48:28Z
user: login: penalosa
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:51:42Z
user: login: emily-shen
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:57:12Z

Review comments:
diff_hunk: @@ -518,8 +518,12 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 		}
 	});
 
-	it("exposes Vectorize bindings", async () => {
-		const name = await helper.vectorize(32, "euclidean");
+	it.only("exposes Vectorize bindings", async () => {
path: packages/wrangler/e2e/dev-with-resources.test.ts
user: login: emily-shen
body: ```suggestion
	it("exposes Vectorize bindings", async () => {
```
created_at: 2025-03-14T13:42:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -572,10 +574,11 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 			`wrangler dev ${flags} --port ${port} --inspector-port ${inspectorPort} --experimental-vectorize-bind-to-prod`
 		);
 		const { url } = await worker.waitForReady();
-		const res = await fetch(url);
+		await fetch(`${url}/insert`);
+		const res = await fetch(`${url}/query`);
path: packages/wrangler/e2e/dev-with-resources.test.ts
user: login: emily-shen
body: can you leave a comment here saying its reading from another run's query?
created_at: 2025-03-14T13:46:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -572,10 +574,11 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 			`wrangler dev ${flags} --port ${port} --inspector-port ${inspectorPort} --experimental-vectorize-bind-to-prod`
 		);
 		const { url } = await worker.waitForReady();
-		const res = await fetch(url);
+		await fetch(`${url}/insert`);
+		const res = await fetch(`${url}/query`);
path: packages/wrangler/e2e/dev-with-resources.test.ts
user: login: emily-shen
body: also how does this work on the first run? or is the idea it'll inevitably fail on the very first run and we just need to rerun?
created_at: 2025-03-14T13:48:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -572,10 +574,11 @@ describe.sequential.each(RUNTIMES)("Bindings: $flags", ({ runtime, flags }) => {
 			`wrangler dev ${flags} --port ${port} --inspector-port ${inspectorPort} --experimental-vectorize-bind-to-prod`
 		);
 		const { url } = await worker.waitForReady();
-		const res = await fetch(url);
+		await fetch(`${url}/insert`);
+		const res = await fetch(`${url}/query`);
path: packages/wrangler/e2e/dev-with-resources.test.ts
user: login: penalosa
body: Yeah, it'll fail on the first run when a new insert is added, but that's very rare
created_at: 2025-03-14T13:51:42Z
author_association: CONTRIBUTOR

PR comments:
user: login: changeset-bot[bot]
created_at: 2025-03-14T13:28:47Z
body: ###  ⚠️  No Changeset found

Latest commit: 7f566fd902dad070f128bebde4f660e20f79c1fb

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. **If these changes should result in a version bump, you need to add a changeset.**

<details><summary>This PR includes no changesets</summary>

  When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

</details>

[Click here to learn what changesets are, and how to add one](https://github.com/changesets/changesets/blob/main/docs/adding-a-changeset.md).

[Click here if you're a maintainer who wants to add a changeset to this PR](https://github.com/cloudflare/workers-sdk/new/penalosa/improve-vectoriz-e2e?filename=.changeset/four-spiders-obey.md&value=---%0A%22wrangler%22%3A%20patch%0A---%0A%0AUse%20a%20shared%20Vectorize%20index%20for%20the%20Vectorize%20E2E%20tests%0A)


author_association: NONE
user: login: github-actions[bot]
created_at: 2025-03-14T13:31:05Z
body: A wrangler prerelease is available for testing. You can install this latest build in your project with:

```sh
npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-wrangler-8492
```

You can reference the automatically updated head of this PR with:

```sh
npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/8492/npm-package-wrangler-8492
```

Or you can use `npx` with this latest build directly:

```sh
npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-wrangler-8492 dev path/to/script.js
```

<details><summary>Additional artifacts:</summary>


cloudflare-workers-bindings-extension:
```sh
wget https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-workers-bindings-extension-8492 -O ./cloudflare-workers-bindings-extension.0.0.0-ve7e3abe1a.vsix && code --install-extension ./cloudflare-workers-bindings-extension.0.0.0-ve7e3abe1a.vsix
```



create-cloudflare:
```sh
npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-create-cloudflare-8492 --no-auto-update
```



@cloudflare/kv-asset-handler:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-kv-asset-handler-8492
```



miniflare:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-miniflare-8492
```



@cloudflare/pages-shared:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-pages-shared-8492
```



@cloudflare/unenv-preset:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-unenv-preset-8492
```



@cloudflare/vite-plugin:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-vite-plugin-8492
```



@cloudflare/vitest-pool-workers:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-vitest-pool-workers-8492
```



@cloudflare/workers-editor-shared:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-workers-editor-shared-8492
```



@cloudflare/workers-shared:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-workers-shared-8492
```



@cloudflare/workflows-shared:
```sh
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/13858075026/npm-package-cloudflare-workflows-shared-8492
```


Note that these links will no longer work once [the GitHub Actions artifact expires](https://docs.github.com/en/organizations/managing-organization-settings/configuring-the-retention-period-for-github-actions-artifacts-and-logs-in-your-organization).

</details>

---

`wrangler@4.0.0` includes the following runtime dependencies:

|Package|Constraint|Resolved|
|-------|----------|--------|
|`miniflare`|workspace:*|4.20250310.0|
|`workerd`|1.20250310.0|1.20250310.0|
|`workerd --version`|1.20250310.0|2025-03-10|

Please ensure constraints are pinned, and `miniflare`/`workerd` minor versions match.
<!-- Sticky Pull Request Comment -->
author_association: CONTRIBUTOR