Repository: vercel/ai
Pull request #5860: chore (automation): enforce `patch` bumps in `.changeset/*.md` files
Author: login: gr2m
Created: 2025-04-20T19:44:23Z | Merged: 2025-04-23T08:22:30Z

<!--
Welcome to contributing to AI SDK! We're excited to see your changes.

We suggest you read the following contributing guide we've created before submitting:

https://github.com/vercel/ai/blob/main/CONTRIBUTING.md
-->

## Background

Closes #5859

## Summary

This pull request adds a new GitHu Actions workflow that verifies `.changelog/*.md` files added in pull request. As per [the versioning guidelines](https://sdk.vercel.ai/docs/migration-guides/versioning), only `patch` bumps are allowed. For minor releases, the `minor-release` label must be added to the pull request.

Here is a preview from my sandbox repository

- success: https://github.com/gr2m/sandbox/actions/runs/14562664486?pr=295

  <details><summary>screenshot</summary><img width="1498" alt="image" src="https://github.com/user-attachments/assets/69289ebc-beaf-47f8-8df4-2d4a076d5c2a" /></details>
  
- error: https://github.com/gr2m/sandbox/actions/runs/14562659340?pr=296

  <details><summary>screenshot</summary><img width="1486" alt="image" src="https://github.com/user-attachments/assets/b303540e-09e1-495a-9796-247ef7698099" /></details>



## Tasks

<!-- Please check if the PR fulfills the following requirements: -->

- [x] Tests for the changes have been added (for bug fixes / features)
- [x] Docs have been added / updated (for bug fixes / features)
- [x] If required, a _patch_ changeset for relevant packages has been added
- [x] You've run `pnpm prettier-fix` to fix any formatting issues

## Future Work

The `minor-release` label needs to be added to the repository. We might also need a `major-release` label. And we might want to document the process for minor/major releases if we haven't already, and mention the need to add the labels.


Linked issue #5859: [maintenance] enforce `patch` in `.changeset/*.md` files from contributors
### Description

Vercel does not follow semantic versioning for `vercel/ai` packages. Instead, it uses `patch` for bug fixes and features. See documentation at https://sdk.vercel.ai/docs/migration-guides/versioning

A `minor` version bump is reserved for "marketing" releases that aggregate improvements since the last minor release, accommodated by blog post.

In the past, contributions slipped through with a changeset using `minor` which resulted in confusion due to a lacking blog post. In order to prevent that from happening again, we want to implement a CI check that enforces the use of `patch`, except for pull requests that intentionally release a new minor release. We could either check if [a migration guide has been added](https://github.com/vercel/ai/pull/5290/files#diff-91799f77862f5738c951936e6af443fb4b1d88afae225eefc54bac8a526d9a10) or look for the presence of a dedicated label
Linked issue comments: user: login: gr2m
created_at: 2025-04-23T16:03:23Z
body: there is a problem, I'm looking into it: https://github.com/vercel/ai/pull/5916#event-17367402153
author_association: COLLABORATOR
user: login: gr2m
created_at: 2025-04-23T19:28:29Z
body: @samdenty caught a problem that occurs when a `.changeset/*` files get removed, but no other ones are added or edited. I reproduced the problem here: https://github.com/gr2m/sandbox/actions/runs/14626462613?pr=299
author_association: COLLABORATOR
user: login: gr2m
created_at: 2025-04-23T19:30:51Z
body: Confirmed the fix: https://github.com/gr2m/sandbox/pull/299/commits/b76b2a695a467d5833a26bf0d6706552fc44a780
author_association: COLLABORATOR

Diff:
diff --git a/.github/workflows/actions/verify-changesets/index.js b/.github/workflows/actions/verify-changesets/index.js
new file mode 100644
index 000000000000..a44d5b652e8e
--- /dev/null
+++ b/.github/workflows/actions/verify-changesets/index.js
@@ -0,0 +1,137 @@
+import fs from 'node:fs/promises';
+
+const BYPASS_LABELS = ['minor', 'major'];
+
+// check if current file is the entry point
+if (import.meta.url.endsWith(process.argv[1])) {
+  // https://docs.github.com/en/webhooks/webhook-events-and-payloads#pull_request
+  const pullRequestEvent = JSON.parse(
+    await fs.readFile(process.env.GITHUB_EVENT_PATH, 'utf-8'),
+  );
+
+  try {
+    const message = await verifyChangesets(
+      pullRequestEvent,
+      process.env,
+      fs.readFile,
+    );
+    await fs.writeFile(
+      process.env.GITHUB_STEP_SUMMARY,
+      `## Changeset verification passed ✅\n\n${message || ''}`,
+    );
+  } catch (error) {
+    // write error to summary
+    console.error(error.message);
+    await fs.writeFile(
+      process.env.GITHUB_STEP_SUMMARY,
+      `## Changeset verification failed ❌
+
+${error.message}`,
+    );
+
+    if (error.path) {
+      await fs.appendFile(
+        process.env.GITHUB_STEP_SUMMARY,
+        `\n\nFile: \`${error.path}\``,
+      );
+    }
+
+    if (error.content) {
+      await fs.appendFile(
+        process.env.GITHUB_STEP_SUMMARY,
+        `\n\n\`\`\`yaml\n${error.content}\n\`\`\``,
+      );
+    }
+
+    process.exit(1);
+  }
+}
+
+export async function verifyChangesets(
+  event,
+  env = process.env,
+  readFile = fs.readFile,
+) {
+  // Skip check if pull request has "minor-release" label
+  const byPassLabel = event.pull_request.labels.find(label =>
+    BYPASS_LABELS.includes(label.name),
+  );
+  if (byPassLabel) {
+    return `Skipping changeset verification - "${byPassLabel.name}" label found`;
+  }
+
+  // Iterate through all changed .changeset/*.md files
+  for (const path of env.CHANGED_FILES.trim().split(' ')) {
+    // ignore README.md file
+    if (path === '.changeset/README.md') continue;
+
+    // Check if the file is a .changeset file
+    if (!/^\.changeset\/[a-z-]+\.md/.test(path)) {
+      throw Object.assign(new Error(`Invalid file - not a .changeset file`), {
+        path,
+      });
+    }
+
+    // find frontmatter
+    const content = await readFile(`../../../../${path}`, 'utf-8');
+    const result = content.match(/---\n([\s\S]+?)\n---/);
+    if (!result) {
+      throw Object.assign(
+        new Error(`Invalid .changeset file - no frontmatter found`),
+        {
+          path,
+          content,
+        },
+      );
+    }
+
+    const [frontmatter] = result;
+
+    // Find version bump by package. `frontmatter` looks like this:
+    //
+    // ```yaml
+    // 'ai': patch
+    // '@ai-sdk/provider': patch
+    // ```
+    const lines = frontmatter.split('\n').slice(1, -1);
+    const versionBumps = {};
+    for (const line of lines) {
+      const [packageName, versionBump] = line.split(':').map(s => s.trim());
+      if (!packageName || !versionBump) {
+        throw Object.assign(
+          new Error(`Invalid .changeset file - invalid frontmatter`, {
+            path,
+            content,
+          }),
+        );
+      }
+
+      // Check if packageName is already set
+      if (versionBumps[packageName]) {
+        throw Object.assign(
+          new Error(
+            `Invalid .changeset file - duplicate package name "${packageName}"`,
+          ),
+          { path, content },
+        );
+      }
+
+      versionBumps[packageName] = versionBump;
+    }
+
+    // check if any of the version bumps are not "patch"
+    const invalidVersionBumps = Object.entries(versionBumps).filter(
+      ([, versionBump]) => versionBump !== 'patch',
+    );
+
+    if (invalidVersionBumps.length > 0) {
+      throw Object.assign(
+        new Error(
+          `Invalid .changeset file - invalid version bump (only "patch" is allowed, see https://sdk.vercel.ai/docs/migration-guides/versioning). To bypass, add one of the following labels: ${BYPASS_LABELS.join(', ')}`,
+        ),
+
+        { path, content },
+      );
+    }
+  }
+}
diff --git a/.github/workflows/actions/verify-changesets/package.json b/.github/workflows/actions/verify-changesets/package.json
new file mode 100644
index 000000000000..bff806df20e3
--- /dev/null
+++ b/.github/workflows/actions/verify-changesets/package.json
@@ -0,0 +1,8 @@
+{
+  "name": "verify-changesets-action",
+  "private": true,
+  "type": "module",
+  "scripts": {
+    "test": "node --test test.js"
+  }
+}
diff --git a/.github/workflows/actions/verify-changesets/test.js b/.github/workflows/actions/verify-changesets/test.js
new file mode 100644
index 000000000000..142312272e08
--- /dev/null
+++ b/.github/workflows/actions/verify-changesets/test.js
@@ -0,0 +1,193 @@
+import assert from 'node:assert';
+import { mock, test } from 'node:test';
+
+import { verifyChangesets } from './index.js';
+
+test('happy path', async () => {
+  const event = {
+    pull_request: {
+      labels: [],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/some-happy-path.md',
+  };
+
+  const readFile = mock.fn(async path => {
+    return `---\nai: patch\n@ai-sdk/provider: patch\n---\n## Test changeset`;
+  });
+
+  await verifyChangesets(event, env, readFile);
+
+  assert.strictEqual(readFile.mock.callCount(), 1);
+  assert.deepStrictEqual(readFile.mock.calls[0].arguments, [
+    '../../../../.changeset/some-happy-path.md',
+    'utf-8',
+  ]);
+});
+
+test('ignores .changeset/README.md', async () => {
+  const event = {
+    pull_request: {
+      labels: [],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/README.md',
+  };
+
+  const readFile = mock.fn(() => {});
+
+  await verifyChangesets(event, env, readFile);
+
+  assert.strictEqual(readFile.mock.callCount(), 0);
+});
+
+test('invalid file - not a .changeset file', async () => {
+  const event = {
+    pull_request: {
+      labels: [],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/not-a-changeset-file.txt',
+  };
+
+  const readFile = mock.fn(() => {});
+
+  await assert.rejects(
+    () => verifyChangesets(event, env, readFile),
+    Object.assign(new Error('Invalid file - not a .changeset file'), {
+      path: '.changeset/not-a-changeset-file.txt',
+    }),
+  );
+
+  assert.strictEqual(readFile.mock.callCount(), 0);
+});
+
+test('invalid .changeset file - no frontmatter', async () => {
+  const event = {
+    pull_request: {
+      labels: [],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/invalid-changeset-file.md',
+  };
+
+  const readFile = mock.fn(async path => {
+    return 'frontmatter missing';
+  });
+  await assert.rejects(
+    () => verifyChangesets(event, env, readFile),
+    Object.assign(new Error('Invalid .changeset file - no frontmatter found'), {
+      path: '.changeset/invalid-changeset-file.md',
+      content: 'frontmatter missing',
+    }),
+  );
+  assert.strictEqual(readFile.mock.callCount(), 1);
+  assert.deepStrictEqual(readFile.mock.calls[0].arguments, [
+    '../../../../.changeset/invalid-changeset-file.md',
+    'utf-8',
+  ]);
+});
+
+test('minor update', async () => {
+  const event = {
+    pull_request: {
+      labels: [],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/patch-update.md .changeset/minor-update.md',
+  };
+
+  const readFile = mock.fn(async path => {
+    if (path.endsWith('patch-update.md')) {
+      return `---\nai: patch\n---\n## Test changeset`;
+    }
+
+    return `---\n@ai-sdk/provider: minor\n---\n## Test changeset`;
+  });
+
+  await assert.rejects(
+    () => verifyChangesets(event, env, readFile),
+    Object.assign(
+      new Error(
+        `Invalid .changeset file - invalid version bump (only "patch" is allowed, see https://sdk.vercel.ai/docs/migration-guides/versioning). To bypass, add one of the following labels: minor, major`,
+      ),
+      {
+        path: '.changeset/minor-update.md',
+        content: '---\n@ai-sdk/provider: minor\n---\n## Test changeset',
+      },
+    ),
+  );
+
+  assert.strictEqual(readFile.mock.callCount(), 2);
+  assert.deepStrictEqual(readFile.mock.calls[0].arguments, [
+    '../../../../.changeset/patch-update.md',
+    'utf-8',
+  ]);
+  assert.deepStrictEqual(readFile.mock.calls[1].arguments, [
+    '../../../../.changeset/minor-update.md',
+    'utf-8',
+  ]);
+});
+
+test('minor update - with "minor" label', async () => {
+  const event = {
+    pull_request: {
+      labels: [
+        {
+          name: 'minor',
+        },
+      ],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/patch-update.md .changeset/minor-update.md',
+  };
+
+  const readFile = mock.fn(async path => {
+    if (path.endsWith('patch-update.md')) {
+      return `---\nai: patch\n---\n## Test changeset`;
+    }
+
+    return `---\n@ai-sdk/provider: minor\n---\n## Test changeset`;
+  });
+
+  const message = await verifyChangesets(event, env, readFile);
+  assert.strictEqual(
+    message,
+    'Skipping changeset verification - "minor" label found',
+  );
+});
+
+test('major update - with "major" label', async () => {
+  const event = {
+    pull_request: {
+      labels: [
+        {
+          name: 'major',
+        },
+      ],
+    },
+  };
+  const env = {
+    CHANGED_FILES: '.changeset/patch-update.md .changeset/major-update.md',
+  };
+
+  const readFile = mock.fn(async path => {
+    if (path.endsWith('patch-update.md')) {
+      return `---\nai: patch\n---\n## Test changeset`;
+    }
+
+    return `---\n@ai-sdk/provider: major\n---\n## Test changeset`;
+  });
+
+  const message = await verifyChangesets(event, env, readFile);
+  assert.strictEqual(
+    message,
+    'Skipping changeset verification - "major" label found',
+  );
+});
diff --git a/.github/workflows/verify-changesets.yml b/.github/workflows/verify-changesets.yml
new file mode 100644
index 000000000000..bd7c3134bf5a
--- /dev/null
+++ b/.github/workflows/verify-changesets.yml
@@ -0,0 +1,38 @@
+# vercel/ai uses https://github.com/changesets/changesets for versioning and changelogs,
+# but is not following semantic versioning. Instead, it uses `patch` for both fixes
+# and features. It uses `minor` for "marketing releases", accompanied by a blog post and migration guide.
+# This workflow verifies that all `.changeset/*.md` files use `patch` unless a `minor-release` label is present.
+name: Verify Changesets
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
+    branches:
+      - main
+    paths:
+      - '.changeset/*.md'
+
+jobs:
+  verify-changesets:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          ref: ${{ github.event.pull_request.head.ref }}
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 'lts/*'
+      - name: get all changed files from .changeset/*.md
+        id: changeset-files
+        run: |
+          echo "changed-files=$(git diff --diff-filter=dr --name-only $BASE_SHA -- '.changeset/*.md' | tr '\n' ' ')" >> $GITHUB_OUTPUT
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+      - name: Verify changesets
+        working-directory: .github/workflows/actions/verify-changesets
+        run: |
+          node index.js
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          CHANGED_FILES: ${{ steps.changeset-files.outputs.changed-files }}


Reviews:
user: login: gr2m
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-20T19:45:16Z
user: login: gr2m
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-20T19:45:35Z
user: login: lgrammel
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-21T10:28:20Z
user: login: lgrammel
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-22T09:25:53Z

Review comments:
diff_hunk: @@ -0,0 +1,139 @@
+import fs from 'node:fs/promises';
+
+// check if current file is the entry point
+if (import.meta.url.endsWith(process.argv[1])) {
path: .github/workflows/actions/verify-changesets/index.js
user: login: gr2m
body: I use this check for testability. The code below is not executed when `.github/workflows/actions/verify-changesets/index.js` is imported in `.github/workflows/actions/verify-changesets/test.js`
created_at: 2025-04-20T19:45:15Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,139 @@
+import fs from 'node:fs/promises';
+
+// check if current file is the entry point
+if (import.meta.url.endsWith(process.argv[1])) {
+  // https://docs.github.com/en/webhooks/webhook-events-and-payloads#pull_request
+  const pullRequestEvent = JSON.parse(
+    await fs.readFile(process.env.GITHUB_EVENT_PATH, 'utf-8'),
+  );
+
+  try {
+    const message = await verifyChangesets(
+      pullRequestEvent,
+      process.env,
+      fs.readFile,
+    );
+    await fs.writeFile(
+      process.env.GITHUB_STEP_SUMMARY,
+      `## Changeset verification passed ✅\n\n${message || ''}`,
+    );
+  } catch (error) {
+    // write error to summary
+    console.error(error.message);
+    console.error('error.path');
+    console.error(error.path);
+    console.error('error.content');
+    console.error(error.content);
path: .github/workflows/actions/verify-changesets/index.js
user: login: gr2m
body: Oops leftover debugging

```suggestion
```
created_at: 2025-04-20T19:45:35Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,38 @@
+# vercel/ai uses https://github.com/changesets/changesets for versioning and changelogs,
+# but is not following semantic versioning. Instead, it uses `patch` for both fixes
+# and features. It uses `minor` for "marketing releases", accompanied by a blog post and migration guide.
+# This workflow verifies that all `.changeset/*.md` files use `patch` unless a `minor-release` label is present.
+name: Verify Changesets
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
path: .github/workflows/verify-changesets.yml
user: login: lgrammel
body: is `labeled` / `unlabeled` needed for changeset verification?
created_at: 2025-04-21T10:28:20Z
author_association: COLLABORATOR

PR comments:
user: login: lgrammel
created_at: 2025-04-21T10:30:26Z
body: How would this behave on the `v5` branch?
author_association: COLLABORATOR
user: login: gr2m
created_at: 2025-04-21T15:59:46Z
body: > How would this behave on the `v5` branch?

It won't do anything for pull requests against `v5`. But I can port it to v5 as well if you like, I just need a decision on the labels for bypassing, see my comment above.
author_association: COLLABORATOR
user: login: lgrammel
created_at: 2025-04-22T09:26:37Z
body: Once it lands we can backport and see how it plays out on `v5`. re labels: `minor`, `major`
author_association: COLLABORATOR
user: login: gr2m
created_at: 2025-04-22T17:22:28Z
body: > `minor`, `major`

Let me update the code to account for these two labels
author_association: COLLABORATOR
user: login: gr2m
created_at: 2025-04-22T17:29:59Z
body: > > `minor`, `major`
> 
> Let me update the code to account for these two labels

done.
author_association: COLLABORATOR