Repository: readmeio/rdme
Pull request #1189: feat(openapi): improvements to circular ref detection, validation errors
Author: login: erunion
Labels: name: enhancement
description: New feature or request
name: dependencies
description: Pull requests that update a dependency file
name: command:openapi
description: Issues pertaining to the `openapi`, `validate`, `reduce`, or `swagger` commands
name: released on @next
name: released
name: backported-to-v9
description: PRs that have been backported to the 9.x channel
Created: 2025-03-05T19:45:12Z | Merged: 2025-03-05T21:15:23Z

## 🧰 Changes

Over the past couple weeks I've been rewriting our OpenAPI parser, `@readme/openapi-parser`, to support a new error leveling system -- you can read up on details on this in https://github.com/readmeio/oas/pull/943. This PR pulls in the fruits of my labor there and across `oas` and `oas-normalize`.

* [x] Upgrades `oas-normalize` to pull in our new parser engine and error handling. The error leveling work is not being used within `rdme` but for things like specification validation that our parser does `rdme` will now be able to surface more than one of these errors at a time. 🎉
  * Support for Node 18 has also been removed.
* [x] Upgrades `oas` and `@readme/oas-examples` to pull in the removal of Node 18 support.

All told this overhaul is mostly a drop-in replacement with the exception that instead of `SyntaxError` and `MissingPointerError` exceptions being thrown from the parser for some invalid API definition cases it will now always be a `ValidationError.`


Diff:
diff --git a/__tests__/commands/openapi/__snapshots__/inspect.test.ts.snap b/__tests__/commands/openapi/__snapshots__/inspect.test.ts.snap
index 3ad9461f7..4d888a9f9 100644
--- a/__tests__/commands/openapi/__snapshots__/inspect.test.ts.snap
+++ b/__tests__/commands/openapi/__snapshots__/inspect.test.ts.snap
@@ -93,8 +93,13 @@ additionalProperties:
  · #/components/schemas/ParameterizedHeader/properties/parameters/additionalProperties
 
 circularRefs:
+ · #/components/schemas/BodyPart/properties/parent
+ · #/components/schemas/MultiPart/properties/bodyParts/items
  · #/components/schemas/MultiPart/properties/parent
- · #/components/schemas/ZoneOffset/properties/rules"
+ · #/components/schemas/ZoneOffset/properties/rules
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetAfter
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetBefore
+ · #/components/schemas/ZoneRules/properties/transitions/items"
 `;
 
 exports[`rdme openapi inspect > feature reports > should generate a report for '@readme/oas-examples/3.0/json/schema-…' (w/ [ 'additionalProperties', …(2) ]) 1`] = `
@@ -114,8 +119,13 @@ additionalProperties:
  · #/components/schemas/ParameterizedHeader/properties/parameters/additionalProperties
 
 circularRefs:
+ · #/components/schemas/BodyPart/properties/parent
+ · #/components/schemas/MultiPart/properties/bodyParts/items
  · #/components/schemas/MultiPart/properties/parent
  · #/components/schemas/ZoneOffset/properties/rules
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetAfter
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetBefore
+ · #/components/schemas/ZoneRules/properties/transitions/items
 
 x-default: You do not use this.
 x-readme.code-samples: You do not use this.
@@ -128,8 +138,13 @@ x-readme.samples-languages: You do not use this."
 exports[`rdme openapi inspect > feature reports > should generate a report for '@readme/oas-examples/3.0/json/schema-…' (w/ [ 'circularRefs', 'readme' ]) 1`] = `
 "
 circularRefs:
+ · #/components/schemas/BodyPart/properties/parent
+ · #/components/schemas/MultiPart/properties/bodyParts/items
  · #/components/schemas/MultiPart/properties/parent
  · #/components/schemas/ZoneOffset/properties/rules
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetAfter
+ · #/components/schemas/ZoneOffsetTransition/properties/offsetBefore
+ · #/components/schemas/ZoneRules/properties/transitions/items
 
 x-default: You do not use this.
 x-readme.code-samples: You do not use this.
diff --git a/__tests__/commands/openapi/__snapshots__/validate.test.ts.snap b/__tests__/commands/openapi/__snapshots__/validate.test.ts.snap
index b49f8b6fb..56083ed60 100644
--- a/__tests__/commands/openapi/__snapshots__/validate.test.ts.snap
+++ b/__tests__/commands/openapi/__snapshots__/validate.test.ts.snap
@@ -232,7 +232,7 @@ exports[`rdme openapi validate > GHA onboarding E2E tests > should reject if use
 
 exports[`rdme openapi validate > error handling > should throw an error if an invalid API definition has many errors 1`] = `
 {
-  "error": [SyntaxError: OpenAPI schema validation failed.
+  "error": [ValidationError: OpenAPI schema validation failed.
 
 REQUIRED must have required property 'url'
 
@@ -262,7 +262,9 @@ ADDITIONAL PROPERTY must NOT have additional properties
 
 exports[`rdme openapi validate > error handling > should throw an error if an invalid OpenAPI 3.0 definition is supplied 1`] = `
 {
-  "error": [MissingPointerError: Token "Error" does not exist.],
+  "error": [ValidationError: API definition schema validation failed.
+
+Missing $ref pointer "#/components/schemas/Error". Token "Error" does not exist.],
   "stderr": "- Validating the API definition located at ./__tests__/__fixtures__/invalid-oas.json...
 ✖ Validating the API definition located at ./__tests__/__fixtures__/invalid-oas.json...
 ",
@@ -272,7 +274,7 @@ exports[`rdme openapi validate > error handling > should throw an error if an in
 
 exports[`rdme openapi validate > error handling > should throw an error if an invalid OpenAPI 3.1 definition is supplied 1`] = `
 {
-  "error": [SyntaxError: OpenAPI schema validation failed.
+  "error": [ValidationError: OpenAPI schema validation failed.
 
 REQUIRED must have required property 'name'
 
@@ -292,7 +294,7 @@ REQUIRED must have required property 'name'
 
 exports[`rdme openapi validate > error handling > should throw an error if an invalid Swagger definition is supplied 1`] = `
 {
-  "error": [SyntaxError: Swagger schema validation failed.
+  "error": [ValidationError: Swagger schema validation failed.
 
 ADDITIONAL PROPERTY must NOT have additional properties
 
diff --git a/package-lock.json b/package-lock.json
index f982e5f34..be8b7bb96 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -15,7 +15,7 @@
         "@oclif/plugin-help": "^6.2.15",
         "@oclif/plugin-not-found": "^3.2.28",
         "@oclif/plugin-warn-if-update-available": "^3.1.19",
-        "@readme/better-ajv-errors": "^2.0.0",
+        "@readme/better-ajv-errors": "^2.3.2",
         "ajv": "^8.17.1",
         "ajv-formats": "^3.0.1",
         "chalk": "^5.3.0",
@@ -26,8 +26,8 @@
         "ignore": "^7.0.0",
         "js-yaml": "^4.1.0",
         "mime-types": "^2.1.35",
-        "oas": "^25.3.0",
-        "oas-normalize": "^12.1.0",
+        "oas": "^26.0.1",
+        "oas-normalize": "^13.0.1",
         "ora": "^8.1.1",
         "prompts": "^2.4.2",
         "semver": "^7.5.3",
@@ -48,7 +48,7 @@
         "@commitlint/config-conventional": "^19.0.3",
         "@oclif/test": "^4.1.0",
         "@readme/eslint-config": "^14.3.0",
-        "@readme/oas-examples": "^5.10.0",
+        "@readme/oas-examples": "^5.19.1",
         "@rollup/plugin-commonjs": "^28.0.0",
         "@rollup/plugin-json": "^6.0.0",
         "@rollup/plugin-node-resolve": "^16.0.0",
@@ -132,11 +132,22 @@
         "node": ">=6.0.0"
       }
     },
-    "node_modules/@apidevtools/swagger-methods": {
-      "version": "3.0.2",
-      "resolved": "https://registry.npmjs.org/@apidevtools/swagger-methods/-/swagger-methods-3.0.2.tgz",
-      "integrity": "sha512-QAkD5kK2b1WfjDS/UQn/qQkbwF31uqRjPTrsCs5ZG9BQGAkjwvqGFjjPqAuzac/IYzpPtRzjCP1WrTuAIjMrXg==",
-      "license": "MIT"
+    "node_modules/@apidevtools/json-schema-ref-parser": {
+      "version": "11.9.3",
+      "resolved": "https://registry.npmjs.org/@apidevtools/json-schema-ref-parser/-/json-schema-ref-parser-11.9.3.tgz",
+      "integrity": "sha512-60vepv88RwcJtSHrD6MjIL6Ta3SOYbgfnkHb+ppAVK+o9mXprRtulx7VlRl3lN3bbvysAfCS7WMVfhUYemB0IQ==",
+      "license": "MIT",
+      "dependencies": {
+        "@jsdevtools/ono": "^7.1.3",
+        "@types/json-schema": "^7.0.15",
+        "js-yaml": "^4.1.0"
+      },
+      "engines": {
+        "node": ">= 16"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/philsturgeon"
+      }
     },
     "node_modules/@aws-crypto/crc32": {
       "version": "5.2.0",
@@ -2963,46 +2974,28 @@
       "integrity": "sha512-/dBh9qw3QhJYqlGwt2I+KUP/lQ6nytdCx3aq+GpMUhibLHF3O7fwoowNcTwlbnwtyJ+TJYTIIrp3oVUlRNx3fA==",
       "license": "MIT"
     },
-    "node_modules/@readme/json-schema-ref-parser": {
-      "version": "1.2.1",
-      "resolved": "https://registry.npmjs.org/@readme/json-schema-ref-parser/-/json-schema-ref-parser-1.2.1.tgz",
-      "integrity": "sha512-FKCnFnpKklBPu8atyXqmSRBPSYlZLdcdbIilX19y0vVFiVthqKV9SQp4GZ8L4rOqSVmjn14uZ4Ono5tZKMr1SQ==",
-      "deprecated": "This package is no longer maintained. Please use `@apidevtools/json-schema-ref-parser` instead.",
-      "license": "MIT",
-      "dependencies": {
-        "@jsdevtools/ono": "^7.1.3",
-        "@types/json-schema": "^7.0.12",
-        "call-me-maybe": "^1.0.1",
-        "js-yaml": "^4.1.0"
-      },
-      "engines": {
-        "node": ">=18"
-      }
-    },
     "node_modules/@readme/oas-examples": {
-      "version": "5.18.2",
-      "resolved": "https://registry.npmjs.org/@readme/oas-examples/-/oas-examples-5.18.2.tgz",
-      "integrity": "sha512-ENAqUaKGLVM7u0KIwuWAqER2D2Fcc8w0EpGOD9BbjVeOnVNQg4gSFvr6zNQtblu5i/D1m3upKqBMeSPIhocXJg==",
+      "version": "5.19.1",
+      "resolved": "https://registry.npmjs.org/@readme/oas-examples/-/oas-examples-5.19.1.tgz",
+      "integrity": "sha512-uFEA5mw2Rj828+duAI6YKDKhwyaS6QbbVqnjKcgUQvR4Qma0Q/NRgYJU+ayPN1CtP60HVwKuxSt2ZjUxdP4QHw==",
       "dev": true,
       "license": "MIT"
     },
     "node_modules/@readme/openapi-parser": {
-      "version": "2.7.0",
-      "resolved": "https://registry.npmjs.org/@readme/openapi-parser/-/openapi-parser-2.7.0.tgz",
-      "integrity": "sha512-P8WSr8WTOxilnT89tcCRKWYsG/II4sAwt1a/DIWub8xTtkrG9cCBBy/IUcvc5X8oGWN82MwcTA3uEkDrXZd/7A==",
+      "version": "3.0.1",
+      "resolved": "https://registry.npmjs.org/@readme/openapi-parser/-/openapi-parser-3.0.1.tgz",
+      "integrity": "sha512-U0Hn3bJRs8ZWn6cylNxLsieJjr5liibzn9JDuWNVvBILK5geEtJDS7S2vMbNViufGGLu08q1dvsbbYIZCSkBzw==",
       "license": "MIT",
       "dependencies": {
-        "@apidevtools/swagger-methods": "^3.0.2",
-        "@jsdevtools/ono": "^7.1.3",
-        "@readme/better-ajv-errors": "^2.0.0",
-        "@readme/json-schema-ref-parser": "^1.2.0",
+        "@apidevtools/json-schema-ref-parser": "^11.9.2",
+        "@readme/better-ajv-errors": "^2.3.2",
         "@readme/openapi-schemas": "^3.1.0",
+        "@types/json-schema": "^7.0.15",
         "ajv": "^8.12.0",
-        "ajv-draft-04": "^1.0.0",
-        "call-me-maybe": "^1.0.1"
+        "ajv-draft-04": "^1.0.0"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
       },
       "peerDependencies": {
         "openapi-types": ">=7"
@@ -13162,12 +13155,12 @@
       }
     },
     "node_modules/oas": {
-      "version": "25.3.0",
-      "resolved": "https://registry.npmjs.org/oas/-/oas-25.3.0.tgz",
-      "integrity": "sha512-98HLqUASE1u4kS8o3O7ZqjT1LG5oNwNjG1oXTsMscYVl2RSW4gMzlrn80dpxjsxpMlVuqPPa/sUS5WFtkk+Xxg==",
+      "version": "26.0.1",
+      "resolved": "https://registry.npmjs.org/oas/-/oas-26.0.1.tgz",
+      "integrity": "sha512-eDaiCqAohUbYvqixseo2IygrfsnkFRNVz3FrKHMze3A8INeCbVNoR2fvSIBEhzwQIEy/nszddA91KBJ6gU3aJw==",
       "license": "MIT",
       "dependencies": {
-        "@readme/json-schema-ref-parser": "^1.2.0",
+        "@readme/openapi-parser": "^3.0.1",
         "@types/json-schema": "^7.0.11",
         "json-schema-merge-allof": "^0.8.1",
         "jsonpath-plus": "^10.0.0",
@@ -13175,10 +13168,10 @@
         "memoizee": "^0.4.16",
         "openapi-types": "^12.1.1",
         "path-to-regexp": "^8.1.0",
-        "remove-undefined-objects": "^5.0.0"
+        "remove-undefined-objects": "^6.0.0"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
       }
     },
     "node_modules/oas-kit-common": {
@@ -13205,19 +13198,19 @@
       }
     },
     "node_modules/oas-normalize": {
-      "version": "12.1.0",
-      "resolved": "https://registry.npmjs.org/oas-normalize/-/oas-normalize-12.1.0.tgz",
-      "integrity": "sha512-YDIkzgiyxV+WunOWtc9zKvGPK37KMhNiCRArSZ2D7olQmcZ3iSmYwZ9NbNj9xgJUAnNVFJJNOFLrXtsR2z/6Yg==",
+      "version": "13.0.1",
+      "resolved": "https://registry.npmjs.org/oas-normalize/-/oas-normalize-13.0.1.tgz",
+      "integrity": "sha512-ZZ4ULY9SAmjnnT5vwvQFdBkL1Sug5IekGbQQh+Tqz3YwligAQfY+Kux7xKcgAgvW77IdUJl+68oo4GeG2D9tFw==",
       "license": "MIT",
       "dependencies": {
-        "@readme/openapi-parser": "^2.7.0",
+        "@readme/openapi-parser": "^3.0.1",
         "@readme/postman-to-openapi": "^4.1.0",
         "js-yaml": "^4.1.0",
         "openapi-types": "^12.1.3",
         "swagger2openapi": "^7.0.8"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
       }
     },
     "node_modules/oas-resolver": {
@@ -15015,9 +15008,9 @@
       }
     },
     "node_modules/remove-undefined-objects": {
-      "version": "5.0.0",
-      "resolved": "https://registry.npmjs.org/remove-undefined-objects/-/remove-undefined-objects-5.0.0.tgz",
-      "integrity": "sha512-DE8C17uIWeHaY4SqIkpQpHXm0MIdYHtIqjieWuh0I2PG8YcZRxFE6pqeEhnRetsrQ7Lu9uvSNQkDbg95NLpvnQ==",
+      "version": "6.0.0",
+      "resolved": "https://registry.npmjs.org/remove-undefined-objects/-/remove-undefined-objects-6.0.0.tgz",
+      "integrity": "sha512-8fR4QQFV2xMKTYXazi1944rpr1f+JOzQu58TgUFi3xDu41fDon5qMXtjJ1/nhquOouTtJ621bKDrhE1IlOSP+A==",
       "license": "MIT",
       "engines": {
         "node": ">=18"
diff --git a/package.json b/package.json
index 88a7ca331..98ba0701d 100644
--- a/package.json
+++ b/package.json
@@ -47,7 +47,7 @@
     "@oclif/plugin-help": "^6.2.15",
     "@oclif/plugin-not-found": "^3.2.28",
     "@oclif/plugin-warn-if-update-available": "^3.1.19",
-    "@readme/better-ajv-errors": "^2.0.0",
+    "@readme/better-ajv-errors": "^2.3.2",
     "ajv": "^8.17.1",
     "ajv-formats": "^3.0.1",
     "chalk": "^5.3.0",
@@ -58,8 +58,8 @@
     "ignore": "^7.0.0",
     "js-yaml": "^4.1.0",
     "mime-types": "^2.1.35",
-    "oas": "^25.3.0",
-    "oas-normalize": "^12.1.0",
+    "oas": "^26.0.1",
+    "oas-normalize": "^13.0.1",
     "ora": "^8.1.1",
     "prompts": "^2.4.2",
     "semver": "^7.5.3",
@@ -77,7 +77,7 @@
     "@commitlint/config-conventional": "^19.0.3",
     "@oclif/test": "^4.1.0",
     "@readme/eslint-config": "^14.3.0",
-    "@readme/oas-examples": "^5.10.0",
+    "@readme/oas-examples": "^5.19.1",
     "@rollup/plugin-commonjs": "^28.0.0",
     "@rollup/plugin-json": "^6.0.0",
     "@rollup/plugin-node-resolve": "^16.0.0",


Reviews:
user: login: erunion
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T19:46:25Z
user: login: erunion
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T19:47:06Z
user: login: erunion
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T19:47:42Z
user: login: kanadgupta
body: just updated the PR title so it's a bit more customer-facing. LGTM!

one quick non-blocking q — should we add a test that illustrates this? 👇🏽 

> The error leveling work is not being used within `rdme` but for things like specification validation that our parser does `rdme` will now be able to surface more than one of these errors at a time. 🎉
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T21:14:02Z

Review comments:
diff_hunk: @@ -93,8 +93,13 @@ additionalProperties:
  · #/components/schemas/ParameterizedHeader/properties/parameters/additionalProperties
 
 circularRefs:
+ · #/components/schemas/BodyPart/properties/parent
path: __tests__/commands/openapi/__snapshots__/inspect.test.ts.snap
user: login: erunion
body: These circular refs weren't being picked up before but `@apidevtools/json-schema-ref-parser` is better at detecting them now with an `onCircular` hook I submitted to them. https://github.com/APIDevTools/json-schema-ref-parser/pull/366
created_at: 2025-03-05T19:46:25Z
author_association: MEMBER
diff_hunk: @@ -172,10 +172,14 @@ ADDITIONAL PROPERTY must NOT have additional properties
   29 |         "summary": "Finds Pets by status",]
 `;
 
-exports[`rdme openapi validate > error handling > should throw an error if an invalid OpenAPI 3.0 definition is supplied 1`] = `[MissingPointerError: Token "Error" does not exist.]`;
+exports[`rdme openapi validate > error handling > should throw an error if an invalid OpenAPI 3.0 definition is supplied 1`] = `
+[ValidationError: API definition schema validation failed.
path: __tests__/commands/openapi/__snapshots__/validate.test.ts.snap
user: login: erunion
body: Slightly nicer error message here with this header being shown instead of dumping the user out with "Token "Error" does not exist."
created_at: 2025-03-05T19:47:05Z
author_association: MEMBER
diff_hunk: @@ -47,7 +47,7 @@
     "@oclif/plugin-help": "^6.2.15",
     "@oclif/plugin-not-found": "^3.2.28",
     "@oclif/plugin-warn-if-update-available": "^3.1.19",
-    "@readme/better-ajv-errors": "^2.0.0",
+    "@readme/better-ajv-errors": "^2.3.2",
path: package.json
user: login: erunion
body: v2.3.2 was already installed but I just bumped it in the package file.
created_at: 2025-03-05T19:47:41Z
author_association: MEMBER

PR comments:
user: login: kanadgupta
created_at: 2025-03-05T19:59:47Z
body: sorry i think #1188 might have caused merge conflicts 🫠
author_association: CONTRIBUTOR
user: login: erunion
created_at: 2025-03-05T21:15:02Z
body: > one quick non-blocking q — should we add a test that illustrates this? 👇🏽
> 
> > The error leveling work is not being used within `rdme` but for things like specification validation that our parser does `rdme` will now be able to surface more than one of these errors at a time. 🎉

Feel like that's already well tested across `oas-normalize` and the parser
author_association: MEMBER
user: login: kanadgupta
created_at: 2025-03-05T21:16:59Z
body: :tada: This PR is included in version 10.2.0-next.5 :tada:

The release is available on:
- [npm package (@next dist-tag)](https://www.npmjs.com/package/rdme/v/10.2.0-next.5)
- `v10.2.0-next.5`
- [GitHub release](https://github.com/readmeio/rdme/releases/tag/v10.2.0-next.5)

Your **[semantic-release](https://github.com/semantic-release/semantic-release)** bot :package::rocket:
author_association: CONTRIBUTOR
user: login: kanadgupta
created_at: 2025-04-29T22:16:14Z
body: :tada: This PR is included in version 10.2.0 :tada:

The release is available on:
- [npm package (@latest dist-tag)](https://www.npmjs.com/package/rdme/v/10.2.0)
- `v10.2.0`
- [GitHub release](https://github.com/readmeio/rdme/releases/tag/v10.2.0)

Your **[semantic-release](https://github.com/semantic-release/semantic-release)** bot :package::rocket:
author_association: CONTRIBUTOR