Repository: alangpierce/sucrase (ts)

Issue:
Support for Flow Enums
Hello - I was testing out the latest version of Sucrase (v3.21.0) and I noticed that it seems to be unable to parse Flow Enums. I end up with an error like:

```
ERROR in ./javascript/devadmin-package/internal/backend-source.jsx
Module build failed (from ./node_modules/@sucrase/webpack-loader/dist/index.js):
SyntaxError: Error transforming /Users/jeresig/khan/webapp/services/static/javascript/devadmin-package/internal/backend-source.jsx: Unexpected token, expected ";" (29:6)
    at unexpected (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/util.js:99:15)
    at semicolon (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/util.js:77:5)
    at flowParseIdentifierStatement (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/plugins/flow.js:806:19)
    at parseIdentifierStatement (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/statement.js:479:40)
    at parseStatementContent (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/statement.js:238:5)
    at parseStatement (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/statement.js:109:3)
    at parseBlockBody (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/statement.js:504:5)
    at parseTopLevel (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/statement.js:85:3)
    at parseFile (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/traverser/index.js:17:35)
    at parse (/Users/jeresig/khan/webapp/services/static/node_modules/sucrase/dist/parser/index.js:26:35)
```

and the code in question:

```
enum Source {
    DevBackend = "use_dev_backend",
    ProdBackend = "use_prod_backend",
}
```

Obviously this would require additional logic to support to turn the code into valid runtime code. It looks like similar logic was implemented to support TypeScript in https://github.com/alangpierce/sucrase/pull/621. Thanks so much for your work on Sucrase, it's a great project!

Pull request description:
Implement Flow enums
Fixes #701

There were a few cases to handle around different types of enums and whethery
they are exported or default export and targeting ESM vs CJS, but overall the
code transform behavior is pretty straightforward. One point of complexity was
that the imports transform needed to opt out of handling for the default export
case so that the enum transform can handle it.

Some caveats in the implementation:
* The snippet `require("flow-enums-runtime")` is not currently configurable like
  it is for the babel plugin. It wouldn't be hard to add this as a new option,
  but I think it's best to leave out config options unless they're needed.
* Flow enums have quite a few constraints which are validated by the babel
  parser and which Sucrase does not validate, such as initializers being
  all-or-nothing, initializers always being literals, and initializers agreeing
  with the enum type. Currently, Sucrase doesn't give a very helpful error when
  these assumptions aren't true, e.g. the transform step can overrun the enum
  body and crash unexpectedly later. This hopefully isn't such a big deal since
  Flow will give a helpful error, but there are probably ways to make the
  Sucrase errors more helpful without sacrificing performance.

Patch:
diff --git a/src/TokenProcessor.ts b/src/TokenProcessor.ts
index 7bd7b86..6143f7c 100644
--- a/src/TokenProcessor.ts
+++ b/src/TokenProcessor.ts
@@ -55,6 +55,10 @@ export default class TokenProcessor {
     return this.identifierNameForToken(this.tokens[index]);
   }
 
+  identifierNameAtRelativeIndex(relativeIndex: number): string {
+    return this.identifierNameForToken(this.tokenAtRelativeIndex(relativeIndex));
+  }
+
   identifierName(): string {
     return this.identifierNameForToken(this.currentToken());
   }
diff --git a/src/transformers/CJSImportTransformer.ts b/src/transformers/CJSImportTransformer.ts
index ca98c41..b10466a 100644
--- a/src/transformers/CJSImportTransformer.ts
+++ b/src/transformers/CJSImportTransformer.ts
@@ -269,8 +269,13 @@ export default class CJSImportTransformer extends Transformer {
       return false;
     }
     if (this.tokens.matches2(tt._export, tt._default)) {
-      this.processExportDefault();
       this.hadDefaultExport = true;
+      if (this.tokens.matches3(tt._export, tt._default, tt._enum)) {
+        // Flow export default enums need some special handling, so handle them
+        // in that tranform rather than this one.
+        return false;
+      }
+      this.processExportDefault();
       return true;
     }
     this.hadNamedExport = true;
diff --git a/src/transformers/FlowTransformer.ts b/src/transformers/FlowTransformer.ts
index 7d37a75..a80deba 100644
--- a/src/transformers/FlowTransformer.ts
+++ b/src/transformers/FlowTransformer.ts
@@ -1,17 +1,182 @@
+import {ContextualKeyword} from "../parser/tokenizer/keywords";
+import {TokenType as tt} from "../parser/tokenizer/types";
 import type TokenProcessor from "../TokenProcessor";
 import type RootTransformer from "./RootTransformer";
 import Transformer from "./Transformer";
 
 export default class FlowTransformer extends Transformer {
-  constructor(readonly rootTransformer: RootTransformer, readonly tokens: TokenProcessor) {
+  constructor(
+    readonly rootTransformer: RootTransformer,
+    readonly tokens: TokenProcessor,
+    readonly isImportsTransformEnabled: boolean,
+  ) {
     super();
   }
 
   process(): boolean {
-    return (
+    if (
       this.rootTransformer.processPossibleArrowParamEnd() ||
       this.rootTransformer.processPossibleAsyncArrowWithTypeParams() ||
       this.rootTransformer.processPossibleTypeRange()
-    );
+    ) {
+      return true;
+    }
+    if (this.tokens.matches1(tt._enum)) {
+      this.processEnum();
+      return true;
+    }
+    if (this.tokens.matches2(tt._export, tt._enum)) {
+      this.processNamedExportEnum();
+      return true;
+    }
+    if (this.tokens.matches3(tt._export, tt._default, tt._enum)) {
+      this.processDefaultExportEnum();
+      return true;
+    }
+    return false;
+  }
+
+  /**
+   * Handle a declaration like:
+   * export enum E ...
+   *
+   * With this imports transform, this becomes:
+   * const E = [[enum]]; exports.E = E;
+   *
+   * otherwise, it becomes:
+   * export const E = [[enum]];
+   */
+  processNamedExportEnum(): void {
+    if (this.isImportsTransformEnabled) {
+      // export
+      this.tokens.removeInitialToken();
+      const enumName = this.tokens.identifierNameAtRelativeIndex(1);
+      this.processEnum();
+      this.tokens.appendCode(` exports.${enumName} = ${enumName};`);
+    } else {
+      this.tokens.copyToken();
+      this.processEnum();
+    }
+  }
+
+  /**
+   * Handle a declaration like:
+   * export default enum E
+   *
+   * With the imports transform, this becomes:
+   * const E = [[enum]]; exports.default = E;
+   *
+   * otherwise, it becomes:
+   * const E = [[enum]]; export default E;
+   */
+  processDefaultExportEnum(): void {
+    // export
+    this.tokens.removeInitialToken();
+    // default
+    this.tokens.removeToken();
+    const enumName = this.tokens.identifierNameAtRelativeIndex(1);
+    this.processEnum();
+    if (this.isImportsTransformEnabled) {
+      this.tokens.appendCode(` exports.default = ${enumName};`);
+    } else {
+      this.tokens.appendCode(` export default ${enumName};`);
+    }
+  }
+
+  /**
+   * Transpile flow enums to invoke the "flow-enums-runtime" library.
+   *
+   * Currently, the transpiled code always uses `require("flow-enums-runtime")`,
+   * but if future flexibility is needed, we could expose a config option for
+   * this string (similar to configurable JSX). Even when targeting ESM, the
+   * default behavior of babel-plugin-transform-flow-enums is to use require
+   * rather than injecting an import.
+   *
+   * Flow enums are quite a bit simpler than TS enums and have some convenient
+   * constraints:
+   * - Element initializers must be either always present or always absent. That
+   *   means that we can use fixed lookahead on the first element (if any) and
+   *   assume that all elements are like that.
+   * - The right-hand side of an element initializer must be a literal value,
+   *   not a complex expression and not referencing other elements. That means
+   *   we can simply copy a single token.
+   *
+   * Enums can be broken up into three basic cases:
+   *
+   * Mirrored enums:
+   * enum E {A, B}
+   *   ->
+   * const E = require("flow-enums-runtime").Mirrored(["A", "B"]);
+   *
+   * Initializer enums:
+   * enum E {A = 1, B = 2}
+   *   ->
+   * const E = require("flow-enums-runtime")({A: 1, B: 2});
+   *
+   * Symbol enums:
+   * enum E of symbol {A, B}
+   *   ->
+   * const E = require("flow-enums-runtime")({A: Symbol("A"), B: Symbol("B")});
+   *
+   * We can statically detect which of the three cases this is by looking at the
+   * "of" declaration (if any) and seeing if the first element has an initializer.
+   * Since the other transform details are so similar between the three cases, we
+   * use a single implementation and vary the transform within processEnumElement
+   * based on case.
+   */
+  processEnum(): void {
+    // enum E -> const E
+    this.tokens.replaceToken("const");
+    this.tokens.copyExpectedToken(tt.name);
+
+    let isSymbolEnum = false;
+    if (this.tokens.matchesContextual(ContextualKeyword._of)) {
+      this.tokens.removeToken();
+      isSymbolEnum = this.tokens.matchesContextual(ContextualKeyword._symbol);
+      this.tokens.removeToken();
+    }
+    const hasInitializers = this.tokens.matches3(tt.braceL, tt.name, tt.eq);
+    this.tokens.appendCode(' = require("flow-enums-runtime")');
+
+    const isMirrored = !isSymbolEnum && !hasInitializers;
+    this.tokens.replaceTokenTrimmingLeftWhitespace(isMirrored ? ".Mirrored([" : "({");
+
+    while (!this.tokens.matches1(tt.braceR)) {
+      // ... is allowed at the end and has no runtime behavior.
+      if (this.tokens.matches1(tt.ellipsis)) {
+        this.tokens.removeToken();
+        break;
+      }
+      this.processEnumElement(isSymbolEnum, hasInitializers);
+      if (this.tokens.matches1(tt.comma)) {
+        this.tokens.copyToken();
+      }
+    }
+
+    this.tokens.replaceToken(isMirrored ? "]);" : "});");
+  }
+
+  /**
+   * Process an individual enum element, producing either an array element or an
+   * object element based on what type of enum this is.
+   */
+  processEnumElement(isSymbolEnum: boolean, hasInitializers: boolean): void {
+    if (isSymbolEnum) {
+      // Symbol enums never have initializers and are expanded to object elements.
+      // A, -> A: Symbol("A"),
+      const elementName = this.tokens.identifierName();
+      this.tokens.copyToken();
+      this.tokens.appendCode(`: Symbol("${elementName}")`);
+    } else if (hasInitializers) {
+      // Initializers are expanded to object elements.
+      // A = 1, -> A: 1,
+      this.tokens.copyToken();
+      this.tokens.replaceTokenTrimmingLeftWhitespace(":");
+      this.tokens.copyToken();
+    } else {
+      // Enum elements without initializers become string literal array elements.
+      // A, -> "A",
+      this.tokens.replaceToken(`"${this.tokens.identifierName()}"`);
+    }
   }
 }
diff --git a/src/transformers/RootTransformer.ts b/src/transformers/RootTransformer.ts
index ce6c3cf..1e167a3 100644
--- a/src/transformers/RootTransformer.ts
+++ b/src/transformers/RootTransformer.ts
@@ -99,7 +99,9 @@ export default class RootTransformer {
     }
 
     if (transforms.includes("flow")) {
-      this.transformers.push(new FlowTransformer(this, tokenProcessor));
+      this.transformers.push(
+        new FlowTransformer(this, tokenProcessor, transforms.includes("imports")),
+      );
     }
     if (transforms.includes("typescript")) {
       this.transformers.push(


Test patch:
diff --git a/test/flow-test.ts b/test/flow-test.ts
index b7d48c3..cdad922 100644
--- a/test/flow-test.ts
+++ b/test/flow-test.ts
@@ -501,4 +501,191 @@ describe("transform flow", () => {
     `,
     );
   });
+
+  it("transforms simple enums with assignments", () => {
+    assertFlowResult(
+      `
+      enum E {
+        A = 1,
+        B = 2,
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime")({
+        A: 1,
+        B: 2,
+      });
+    `,
+    );
+  });
+
+  it("transforms enums without assignments", () => {
+    assertFlowResult(
+      `
+      enum E {
+        A,
+        B,
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B",
+      ]);
+    `,
+    );
+  });
+
+  it("transforms symbol enums", () => {
+    assertFlowResult(
+      `
+      enum E of symbol {
+        A,
+        B
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime")({
+        A: Symbol("A"),
+        B: Symbol("B")
+      });
+    `,
+    );
+  });
+
+  it("transforms number enums", () => {
+    assertFlowResult(
+      `
+      enum E of number {
+        A = 1,
+        B = 2
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime")({
+        A: 1,
+        B: 2
+      });
+    `,
+    );
+  });
+
+  it("transforms empty enums as mirrored with empty array", () => {
+    assertFlowResult(
+      `
+      enum E {
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime").Mirrored([
+      ]);
+    `,
+    );
+  });
+
+  it("transforms mirrored enums with ...", () => {
+    assertFlowResult(
+      `
+      enum E {
+        A,
+        B,
+        ...
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B",
+
+      ]);
+    `,
+    );
+  });
+
+  it("transforms non-mirrored enums with ...", () => {
+    assertFlowResult(
+      `
+      enum E {
+        A = 1,
+        B = 2,
+        ...
+      }
+    `,
+      `"use strict";
+      const E = require("flow-enums-runtime")({
+        A: 1,
+        B: 2,
+
+      });
+    `,
+    );
+  });
+
+  it("handles ESM enum named exports", () => {
+    assertFlowESMResult(
+      `
+      export enum E {
+        A,
+        B
+      }
+    `,
+      `
+      export const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B"
+      ]);
+    `,
+    );
+  });
+
+  it("handles CJS enum named exports", () => {
+    assertFlowResult(
+      `
+      export enum E {
+        A,
+        B
+      }
+    `,
+      `"use strict";${ESMODULE_PREFIX}
+       const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B"
+      ]); exports.E = E;
+    `,
+    );
+  });
+
+  it("handles ESM enum default exports", () => {
+    assertFlowESMResult(
+      `
+      export default enum E {
+        A,
+        B
+      }
+    `,
+      `
+       const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B"
+      ]); export default E;
+    `,
+    );
+  });
+
+  it("handles CJS enum default exports", () => {
+    assertFlowResult(
+      `
+      export default enum E {
+        A,
+        B
+      }
+    `,
+      `"use strict";${ESMODULE_PREFIX}
+       const E = require("flow-enums-runtime").Mirrored([
+        "A",
+        "B"
+      ]); exports.default = E;
+    `,
+    );
+  });
 });