Repository: Khan/perseus
Pull request #2487: Extend parseAndMigratePerseusItem to accept an object as well as string
Author: login: jeremywiebe
Labels: name: olc-5.0.4b96c
Created: 2025-05-14T22:16:16Z | Merged: 2025-05-15T21:36:46Z

## Summary:

The `parseAndMigratePerseusItem` originally accepted only a `string` and started the process by `JSON.parse()`ing it. As we start building more things that need to parse perseus items, it's becoming clear we want to be able to parse objects that are already deserialized. 

So this PR updates the function to accept either a `string` or `object`. 

Issue: LEMS-3054

## Test plan:

`pnpm test`


Diff:
diff --git a/.changeset/seven-toys-lay.md b/.changeset/seven-toys-lay.md
new file mode 100644
index 00000000000..9a82f10f697
--- /dev/null
+++ b/.changeset/seven-toys-lay.md
@@ -0,0 +1,5 @@
+---
+"@khanacademy/perseus-core": minor
+---
+
+Extend parseAndMigratePerseusItem so that it accepts either a JS object or a string (instead of only a string). When an object is passed, it skips calling `JSON.parse()`.
diff --git a/packages/perseus-core/src/parse-perseus-json/index.ts b/packages/perseus-core/src/parse-perseus-json/index.ts
index 0dffdf50f2e..2eaa128696e 100644
--- a/packages/perseus-core/src/parse-perseus-json/index.ts
+++ b/packages/perseus-core/src/parse-perseus-json/index.ts
@@ -40,9 +40,9 @@ export type ParseFailureDetail = {
 };
 
 /**
- * Parses a PerseusItem from a JSON string, migrates old formats to the latest
- * schema, and runtime-typechecks the result. Use this to parse assessmentItem
- * data.
+ * Parses a PerseusItem from a plain object or JSON string, migrates old
+ * formats to the latest schema, and runtime-typechecks the result. Use this to
+ * parse assessmentItem data.
  *
  * @returns a {@link Result} of the parsed PerseusItem. If the result is a
  * failure, it will contain an error message describing where in the tree
@@ -50,10 +50,10 @@ export type ParseFailureDetail = {
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusItem(
-    json: string,
+    data: unknown,
 ): Result<PerseusItem, ParseFailureDetail> {
     throwErrorIfCheatingDetected();
-    const object: unknown = JSON.parse(json);
+    const object: unknown = typeof data === "string" ? JSON.parse(data) : data;
     const result = parse(object, migrateAndTypecheckPerseusItem);
     if (isFailure(result)) {
         return failure({message: result.detail, invalidObject: object});
@@ -62,8 +62,9 @@ export function parseAndMigratePerseusItem(
 }
 
 /**
- * Parses a PerseusArticle from a JSON string, migrates old formats to the
- * latest schema, and runtime-typechecks the result.
+ * Parses a PerseusArticle from a plain object (or array) or JSON string,
+ * migrates old formats to the latest schema, and runtime-typechecks the
+ * result.
  *
  * @returns a {@link Result} of the parsed PerseusArticle. If the result is a
  * failure, it will contain an error message describing where in the tree
@@ -71,10 +72,10 @@ export function parseAndMigratePerseusItem(
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusArticle(
-    json: string,
+    data: unknown,
 ): Result<PerseusArticle, ParseFailureDetail> {
     throwErrorIfCheatingDetected();
-    const object: unknown = JSON.parse(json);
+    const object: unknown = typeof data === "string" ? JSON.parse(data) : data;
     const result = parse(object, migrateAndTypecheckPerseusArticle);
     if (isFailure(result)) {
         return failure({message: result.detail, invalidObject: object});
diff --git a/packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts b/packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
index 281d2c716e1..9d3898ba8d4 100644
--- a/packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
+++ b/packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
@@ -8,7 +8,7 @@ import {
 } from "./index";
 
 describe("parseAndMigratePerseusItem", () => {
-    it("should parse JSON", () => {
+    it("parses a JSON string", () => {
         const result = parseAndMigratePerseusItem(
             `{
                 "answerArea": {},
@@ -27,7 +27,24 @@ describe("parseAndMigratePerseusItem", () => {
         );
     });
 
-    it("returns an error given an invalid PerseusItem", () => {
+    it("parses an object", () => {
+        const result = parseAndMigratePerseusItem({
+            answerArea: {},
+            hints: [],
+            question: {
+                content: "this is the question content",
+                widgets: {},
+                images: {},
+            },
+        });
+
+        assertSuccess(result);
+        expect(result.value.question.content).toBe(
+            "this is the question content",
+        );
+    });
+
+    it("returns an error given an invalid PerseusItem string", () => {
         const result = parseAndMigratePerseusItem(`{"question": "bad value"}`);
 
         assertFailure(result);
@@ -36,6 +53,17 @@ describe("parseAndMigratePerseusItem", () => {
         );
     });
 
+    it("returns an error given an invalid PerseusItem object", () => {
+        const result = parseAndMigratePerseusItem({
+            question: "bad value",
+        });
+
+        assertFailure(result);
+        expect(result.detail.message).toContain(
+            `At (root).question -- expected object, but got "bad value"`,
+        );
+    });
+
     it("returns the invalid object along with the error", () => {
         const result = parseAndMigratePerseusItem(`{"question": "bad value"}`);
 
@@ -58,7 +86,7 @@ describe("parseAndMigratePerseusItem", () => {
 });
 
 describe("parseAndMigratePerseusArticle", () => {
-    it("parses a single renderer", () => {
+    it("parses a single renderer from a string", () => {
         const result = parseAndMigratePerseusArticle(
             `{"content": "", "widgets": {}}`,
         );
@@ -72,7 +100,22 @@ describe("parseAndMigratePerseusArticle", () => {
         );
     });
 
-    it("parses an array of renderers", () => {
+    it("parses a single renderer", () => {
+        const result = parseAndMigratePerseusArticle({
+            content: "",
+            widgets: {},
+        });
+
+        expect(result).toEqual(
+            success({
+                content: "",
+                widgets: {},
+                images: {},
+            }),
+        );
+    });
+
+    it("parses an array of renderers from a string", () => {
         const result = parseAndMigratePerseusArticle(
             `[{"content": "one"}, {"content": "two"}]`,
         );
@@ -84,7 +127,20 @@ describe("parseAndMigratePerseusArticle", () => {
         );
     });
 
-    it("fails given invalid data", () => {
+    it("parses an array of renderers", () => {
+        const result = parseAndMigratePerseusArticle([
+            {content: "one"},
+            {content: "two"},
+        ]);
+        expect(result).toEqual(
+            success([
+                {content: "one", widgets: {}, images: {}},
+                {content: "two", widgets: {}, images: {}},
+            ]),
+        );
+    });
+
+    it("fails given an invalid data string", () => {
         const result = parseAndMigratePerseusArticle("[9]");
 
         assertFailure(result);
@@ -93,6 +149,15 @@ describe("parseAndMigratePerseusArticle", () => {
         );
     });
 
+    it("fails given an invalid data ", () => {
+        const result = parseAndMigratePerseusArticle([9]);
+
+        assertFailure(result);
+        expect(result.detail.message).toEqual(
+            "At (root)[0] -- expected object, but got 9",
+        );
+    });
+
     it("returns the invalid object along with the error", () => {
         const result = parseAndMigratePerseusArticle("[9]");
 


Reviews:
user: login: benchristel
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T23:22:03Z
user: login: benchristel
body: Another thought: since we also have `parseAndMigratePerseusArticle`, should we make a similar change to that function for consistency? I'm pretty sure we're going to want it.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T23:29:07Z
user: login: benchristel
body: I left a few minor suggestions inline, but overall this LGTM!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-15T20:50:31Z
user: login: jeremywiebe
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T21:25:14Z
user: login: jeremywiebe
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T21:28:51Z
user: login: benchristel
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T21:39:07Z

Review comments:
diff_hunk: @@ -40,20 +40,20 @@ export type ParseFailureDetail = {
 };
 
 /**
- * Parses a PerseusItem from a JSON string, migrates old formats to the latest
- * schema, and runtime-typechecks the result. Use this to parse assessmentItem
- * data.
+ * Parses a PerseusItem from a plain object or JSON string, migrates old
+ * formats to the latest schema, and runtime-typechecks the result. Use this to
+ * parse assessmentItem data.
  *
  * @returns a {@link Result} of the parsed PerseusItem. If the result is a
  * failure, it will contain an error message describing where in the tree
  * parsing failed.
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusItem(
-    json: string,
+    json: string | object,
 ): Result<PerseusItem, ParseFailureDetail> {
     throwErrorIfCheatingDetected();
-    const object: unknown = JSON.parse(json);
+    const object = typeof json === "string" ? JSON.parse(json) : json;
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: I think `object` should be explicitly typed as `unknown` for safety — `JSON.parse` returns `any`.

```suggestion
    const object: unknown = typeof json === "string" ? JSON.parse(json) : json;
```
created_at: 2025-05-14T23:18:31Z
author_association: MEMBER
diff_hunk: @@ -40,20 +40,20 @@ export type ParseFailureDetail = {
 };
 
 /**
- * Parses a PerseusItem from a JSON string, migrates old formats to the latest
- * schema, and runtime-typechecks the result. Use this to parse assessmentItem
- * data.
+ * Parses a PerseusItem from a plain object or JSON string, migrates old
+ * formats to the latest schema, and runtime-typechecks the result. Use this to
+ * parse assessmentItem data.
  *
  * @returns a {@link Result} of the parsed PerseusItem. If the result is a
  * failure, it will contain an error message describing where in the tree
  * parsing failed.
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusItem(
-    json: string,
+    json: string | object,
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: Once the object's parsed, it's no longer JSON. Maybe `data` would be a suitably abstract name?

Also, I think the type of this parameter should be `unknown`, since it's likely that this function will be composed with `JSON.parse`. I'm imagining a usage like this:

```ts
/**
 * Typesafe wrapper around `JSON.parse()`, since `JSON.parse`
 * returns `any`.
 */
function parseJSON(document: string): unknown {
  return JSON.parse(document);
}

parseAndMigratePerseusItem(parseJSON(data))
```

```suggestion
    data: unknown,
```
created_at: 2025-05-14T23:18:36Z
author_association: MEMBER
diff_hunk: @@ -27,6 +27,23 @@ describe("parseAndMigratePerseusItem", () => {
         );
     });
 
+    it("should parse an object", () => {
+        const result = parseAndMigratePerseusItem({
+            answerArea: {},
+            hints: [],
+            question: {
+                content: "this is the question content",
+                widgets: {},
+                images: {},
+            },
+        });
+
+        assertSuccess(result);
+        expect(result.value.question.content).toBe(
+            "this is the question content",
+        );
+    });
path: packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
user: login: benchristel
body: To show that `parseAndMigratePerseusItem` isn't just the identity function, maybe we could assert that the parsing logic actually happens — i.e. we should have a test for an error case.
created_at: 2025-05-14T23:20:48Z
author_association: MEMBER
diff_hunk: @@ -62,19 +66,20 @@ export function parseAndMigratePerseusItem(
 }
 
 /**
- * Parses a PerseusArticle from a JSON string, migrates old formats to the
- * latest schema, and runtime-typechecks the result.
+ * Parses a PerseusArticle from a plain object (or array) or JSON string,
+ * migrates old formats to the latest schema, and runtime-typechecks the
+ * result.
  *
  * @returns a {@link Result} of the parsed PerseusArticle. If the result is a
  * failure, it will contain an error message describing where in the tree
  * parsing failed.
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusArticle(
-    json: string,
+    json: string | unknown,
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: `unknown` is a supertype of `string`, so this is equivalent to

```suggestion
    json: unknown,
```

If we want to document the expected input values and the resulting behavior, I think the place to do that is in the doc comment (which you've already done) and in the tests (which you've already done).
created_at: 2025-05-15T20:37:34Z
author_association: MEMBER
diff_hunk: @@ -39,21 +39,25 @@ export type ParseFailureDetail = {
     invalidObject: unknown;
 };
 
+function maybeParseJson(json: string | unknown): unknown {
+    return typeof json === "string" ? JSON.parse(json) : json;
+}
+
 /**
- * Parses a PerseusItem from a JSON string, migrates old formats to the latest
- * schema, and runtime-typechecks the result. Use this to parse assessmentItem
- * data.
+ * Parses a PerseusItem from a plain object or JSON string, migrates old
+ * formats to the latest schema, and runtime-typechecks the result. Use this to
+ * parse assessmentItem data.
  *
  * @returns a {@link Result} of the parsed PerseusItem. If the result is a
  * failure, it will contain an error message describing where in the tree
  * parsing failed.
  * @throws SyntaxError if the argument is not well-formed JSON.
  */
 export function parseAndMigratePerseusItem(
-    json: string,
+    json: string | unknown,
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: ```suggestion
    json: unknown,
```
created_at: 2025-05-15T20:37:55Z
author_association: MEMBER
diff_hunk: @@ -39,21 +39,25 @@ export type ParseFailureDetail = {
     invalidObject: unknown;
 };
 
+function maybeParseJson(json: string | unknown): unknown {
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: Could we name this something other than `maybeParseJson`? The problem with `maybe` is that it doesn't say:

- when the behavior suggested by the name doesn't happen. I.e. when do we not parse?
- what happens when we don't do the behavior. Do we return null? Return the input unchanged? Throw an exception?

I'd prefer a name like `ensureJsonParsed` to express the idea that this function is supposed to be idempotent. Although, it isn't really idempotent; it's totally possible for `JSON.parse` to return a string, e.g. `JSON.parse('""')`, in which case running the result through the function again would _not_ be a no-op (for the specific example I've given, it would throw an exception). The only reason we can treat it as idempotent is because we know that Perseus JSON isn't supposed to have a string literal at the top level. That assumption implicitly couples this function to its context.

Overall, I'd say this function feels like an [Idea Fragment](https://github.com/benchristel/refactoring-workshop/blob/master/docs/smells/idea-fragment.md), and the code would probably be more readable overall if it were inlined.
created_at: 2025-05-15T20:38:58Z
author_association: MEMBER
diff_hunk: @@ -36,6 +53,17 @@ describe("parseAndMigratePerseusItem", () => {
         );
     });
 
+    it("returns an error given an invalid PerseusItem object", () => {
+        const result = parseAndMigratePerseusItem({
+            question: "bad value",
+        } as any);
path: packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
user: login: benchristel
body: Do we need to cast to `any` here? It seems like this object should be assignable to `unknown`.
created_at: 2025-05-15T20:48:12Z
author_association: MEMBER
diff_hunk: @@ -36,6 +53,17 @@ describe("parseAndMigratePerseusItem", () => {
         );
     });
 
+    it("returns an error given an invalid PerseusItem object", () => {
+        const result = parseAndMigratePerseusItem({
+            question: "bad value",
+        } as any);
path: packages/perseus-core/src/parse-perseus-json/parse-perseus-json.test.ts
user: login: jeremywiebe
body: The `as any` here was because I _thought_ that I needed `@ts-expect-error`. However, the input argument isn't a `PerseusItem` and so doesn't need to conform to any structure! I've removed it. I was sure I'd seen a Typescript error here, but when I remove it, it's fine. Thanks for catching that. 
created_at: 2025-05-15T21:25:14Z
author_association: COLLABORATOR
diff_hunk: @@ -39,21 +39,25 @@ export type ParseFailureDetail = {
     invalidObject: unknown;
 };
 
+function maybeParseJson(json: string | unknown): unknown {
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: jeremywiebe
body: I'd never encountered the idea of an Idea Fragment before, that's a useful concept, thanks. I've inlined the code as it's really small. 
created_at: 2025-05-15T21:28:51Z
author_association: COLLABORATOR
diff_hunk: @@ -39,21 +39,25 @@ export type ParseFailureDetail = {
     invalidObject: unknown;
 };
 
+function maybeParseJson(json: string | unknown): unknown {
path: packages/perseus-core/src/parse-perseus-json/index.ts
user: login: benchristel
body: > I'd never encountered the idea of an Idea Fragment before

It's pretty obscure.
created_at: 2025-05-15T21:39:07Z
author_association: MEMBER

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-14T22:17:33Z
body: **Size Change:** +17 B (0%) 

**Total Size:** 467 kB

| Filename | Size | Change |
| :--- | :---: | :---: |
| `packages/perseus-core/dist/es/index.js` | 19 kB | +17 B (+0.09%) |

<details><summary>ℹ️ <strong>View Unchanged</strong></summary>

| Filename | Size |
| :--- | :---: |
| `packages/kas/dist/es/index.js` | 20.7 kB |
| `packages/keypad-context/dist/es/index.js` | 1 kB |
| `packages/kmath/dist/es/index.js` | 5.98 kB |
| `packages/math-input/dist/es/index.js` | 98.6 kB |
| `packages/math-input/dist/es/strings.js` | 1.61 kB |
| `packages/perseus-editor/dist/es/index.js` | 88.6 kB |
| `packages/perseus-linter/dist/es/index.js` | 7.05 kB |
| `packages/perseus-score/dist/es/index.js` | 9.04 kB |
| `packages/perseus-utils/dist/es/index.js` | 403 B |
| `packages/perseus/dist/es/index.js` | 200 kB |
| `packages/perseus/dist/es/strings.js` | 7.49 kB |
| `packages/pure-markdown/dist/es/index.js` | 1.22 kB |
| `packages/simple-markdown/dist/es/index.js` | 6.71 kB |

</details>



<a href="https://github.com/preactjs/compressed-size-action"><sub>compressed-size-action</sub></a>
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-14T22:19:54Z
body: # npm Snapshot: Published

Good news!! We've packaged up the latest commit from this PR (24814101) and published it to npm. You
can install it using the tag `PR2487`.

Example:
```sh
pnpm add @khanacademy/perseus@PR2487
```

If you are working in Khan Academy's webapp, you can run the below command. 
If this is for backend code, change the static flag (-s) from y/yes to n/no.
```sh
./dev/tools/bump_perseus_version.js -s y -t PR2487
```
author_association: CONTRIBUTOR
user: login: jeremywiebe
created_at: 2025-05-15T20:09:21Z
body: > Another thought: since we also have `parseAndMigratePerseusArticle`, should we make a similar change to that function for consistency? I'm pretty sure we're going to want it.

Good call. I've updated `parseAndMigratePerseusArticle` and added tests for it too.
author_association: COLLABORATOR