Repository: alangpierce/sucrase
Pull request #739 (2022-08-16 20:42:04)

Implement parser changes for the JSX automatic runtime
Progress toward #585

This change extends the parser to detect two JSX cases that are needed for the
automatic runtime. The new code is run unconditionally, even in the old
transform, since flagging it would have its own overhead and complexity.

The new JSX transform needs to distinguish children being static or non-static,
where static is equivalent to having at least two comma-separated children
emitted by the old transform or any child being a spread child. The main
challenge in getting this right is that JSX whitespace-trimming will sometimes
completely remove all content from a text range, in which case it shouldn't
count as a child for the purposes of counting the number of children.
Fortunately, there is a relatively simple algorithm to detect if a text range
is empty: it's empty if and only if it is entirely whitespace and has at least
one newline. To identify these "empty text" regions, I added a new token type
`jsxEmptyText` that is treated the same except for the purposes of counting
children. In the future, it likely is reasonable to not treat such a region as a
token in the first place.

The new JSX transform also needs to detect the pattern of a key appearing after
a prop spread. We don't do keyword detection on JSX prop names, so instead I
manually detect the name "key", but only if we have already seen a prop spread.

codecov[bot]: # [Codecov](https://codecov.io/gh/alangpierce/sucrase/pull/739?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce) Report
> Merging [#739](https://codecov.io/gh/alangpierce/sucrase/pull/739?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce) (4d46bda) into [main](https://codecov.io/gh/alangpierce/sucrase/commit/36b82ba0285ebab0ab8f09c0d553925224a4f94c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce) (36b82ba) will **increase** coverage by `0.06%`.
> The diff coverage is `96.49%`.

```diff
@@            Coverage Diff             @@
##             main     #739      +/-   ##
==========================================
+ Coverage   87.17%   87.23%   +0.06%     
==========================================
  Files          54       54              
  Lines        5722     5751      +29     
  Branches     1349     1357       +8     
==========================================
+ Hits         4988     5017      +29     
  Misses        466      466              
  Partials      268      268              
```


| [Impacted Files](https://codecov.io/gh/alangpierce/sucrase/pull/739?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce) | Coverage Δ | |
|---|---|---|
| [src/parser/traverser/expression.ts](https://codecov.io/gh/alangpierce/sucrase/pull/739/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce#diff-c3JjL3BhcnNlci90cmF2ZXJzZXIvZXhwcmVzc2lvbi50cw==) | `88.41% <0.00%> (ø)` | |
| [src/transformers/JSXTransformer.ts](https://codecov.io/gh/alangpierce/sucrase/pull/739/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce#diff-c3JjL3RyYW5zZm9ybWVycy9KU1hUcmFuc2Zvcm1lci50cw==) | `97.16% <0.00%> (ø)` | |
| [src/parser/plugins/jsx/index.ts](https://codecov.io/gh/alangpierce/sucrase/pull/739/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce#diff-c3JjL3BhcnNlci9wbHVnaW5zL2pzeC9pbmRleC50cw==) | `92.26% <100.00%> (+1.16%)` | :arrow_up: |
| [src/parser/tokenizer/index.ts](https://codecov.io/gh/alangpierce/sucrase/pull/739/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce#diff-c3JjL3BhcnNlci90b2tlbml6ZXIvaW5kZXgudHM=) | `86.28% <100.00%> (+0.17%)` | :arrow_up: |
| [src/parser/util/charcodes.ts](https://codecov.io/gh/alangpierce/sucrase/pull/739/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce#diff-c3JjL3BhcnNlci91dGlsL2NoYXJjb2Rlcy50cw==) | `98.16% <100.00%> (+0.01%)` | :arrow_up: |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Alan+Pierce)

github-actions[bot]: ## Benchmark results
**Before this PR:** 397.7 thousand lines per second
**After this PR:**  399.5 thousand lines per second

**Measured change:** 0.46% faster (0.77% slower to 0.7% faster)
**Summary:** Likely no significant difference

Patch:
diff --git a/generator/generateTokenTypes.ts b/generator/generateTokenTypes.ts
index b260f4d..3caf2e6 100644
--- a/generator/generateTokenTypes.ts
+++ b/generator/generateTokenTypes.ts
@@ -112,6 +112,7 @@ const types = {
 
   jsxName: new TokenType("jsxName"),
   jsxText: new TokenType("jsxText"),
+  jsxEmptyText: new TokenType("jsxEmptyText"),
   jsxTagStart: new TokenType("jsxTagStart", {startsExpr}),
   jsxTagEnd: new TokenType("jsxTagEnd"),
   typeParameterStart: new TokenType("typeParameterStart", {startsExpr}),
diff --git a/src/parser/plugins/jsx/index.ts b/src/parser/plugins/jsx/index.ts
index fee7f34..357d578 100644
--- a/src/parser/plugins/jsx/index.ts
+++ b/src/parser/plugins/jsx/index.ts
@@ -3,6 +3,7 @@ import {
   finishToken,
   getTokenFromCode,
   IdentifierRole,
+  JSXRole,
   match,
   next,
   skipSpace,
@@ -16,34 +17,58 @@ import {charCodes} from "../../util/charcodes";
 import {IS_IDENTIFIER_CHAR, IS_IDENTIFIER_START} from "../../util/identifier";
 import {tsTryParseJSXTypeArgument} from "../typescript";
 
-// Reads inline JSX contents token.
+/**
+ * Read token with JSX contents.
+ *
+ * In addition to detecting jsxTagStart and also regular tokens that might be
+ * part of an expression, this code detects the start and end of text ranges
+ * within JSX children. In order to properly count the number of children, we
+ * distinguish jsxText from jsxEmptyText, which is a text range that simplifies
+ * to the empty string after JSX whitespace trimming.
+ *
+ * It turns out that a JSX text range will simplify to the empty string if and
+ * only if both of these conditions hold:
+ * - The range consists entirely of whitespace characters (only counting space,
+ *   tab, \r, and \n).
+ * - The range has at least one newline.
+ * This can be proven by analyzing any implementation of whitespace trimming,
+ * e.g. formatJSXTextLiteral in Sucrase or cleanJSXElementLiteralChild in Babel.
+ */
 function jsxReadToken(): void {
-  for (;;) {
+  let sawNewline = false;
+  let sawNonWhitespace = false;
+  while (true) {
     if (state.pos >= input.length) {
       unexpected("Unterminated JSX contents");
       return;
     }
 
     const ch = input.charCodeAt(state.pos);
-
-    switch (ch) {
-      case charCodes.lessThan:
-      case charCodes.leftCurlyBrace:
-        if (state.pos === state.start) {
-          if (ch === charCodes.lessThan) {
-            state.pos++;
-            finishToken(tt.jsxTagStart);
-            return;
-          }
-          getTokenFromCode(ch);
+    if (ch === charCodes.lessThan || ch === charCodes.leftCurlyBrace) {
+      if (state.pos === state.start) {
+        if (ch === charCodes.lessThan) {
+          state.pos++;
+          finishToken(tt.jsxTagStart);
           return;
         }
-        finishToken(tt.jsxText);
+        getTokenFromCode(ch);
         return;
+      }
+      if (sawNewline && !sawNonWhitespace) {
+        finishToken(tt.jsxEmptyText);
+      } else {
+        finishToken(tt.jsxText);
+      }
+      return;
+    }
 
-      default:
-        state.pos++;
+    // This is part of JSX text.
+    if (ch === charCodes.lineFeed) {
+      sawNewline = true;
+    } else if (ch !== charCodes.space && ch !== charCodes.carriageReturn && ch !== charCodes.tab) {
+      sawNonWhitespace = true;
     }
+    state.pos++;
   }
 }
 
@@ -116,7 +141,7 @@ function jsxParseAttributeValue(): void {
   switch (state.type) {
     case tt.braceL:
       next();
-      jsxParseExpressionContainer();
+      parseExpression();
       nextJSXTagToken();
       return;
 
@@ -134,10 +159,6 @@ function jsxParseAttributeValue(): void {
   }
 }
 
-function jsxParseEmptyExpression(): void {
-  // Do nothing.
-}
-
 // Parse JSX spread child, after already processing the {
 // Does not parse the closing }
 function jsxParseSpreadChild(): void {
@@ -145,36 +166,10 @@ function jsxParseSpreadChild(): void {
   parseExpression();
 }
 
-// Parses JSX expression enclosed into curly brackets, after already processing the {
-// Does not parse the closing }
-function jsxParseExpressionContainer(): void {
-  if (match(tt.braceR)) {
-    jsxParseEmptyExpression();
-  } else {
-    parseExpression();
-  }
-}
-
-// Parses following JSX attribute name-value pair.
-function jsxParseAttribute(): void {
-  if (eat(tt.braceL)) {
-    expect(tt.ellipsis);
-    parseMaybeAssign();
-    // }
-    nextJSXTagToken();
-    return;
-  }
-  jsxParseNamespacedName(IdentifierRole.ObjectKey);
-  if (match(tt.eq)) {
-    nextJSXTagToken();
-    jsxParseAttributeValue();
-  }
-}
-
 // Parses JSX opening tag starting after "<".
 // Returns true if the tag was self-closing.
 // Does not parse the last token.
-function jsxParseOpeningElement(): boolean {
+function jsxParseOpeningElement(initialTokenIndex: number): boolean {
   if (match(tt.jsxTagEnd)) {
     // This is an open-fragment.
     return false;
@@ -183,8 +178,30 @@ function jsxParseOpeningElement(): boolean {
   if (isTypeScriptEnabled) {
     tsTryParseJSXTypeArgument();
   }
+  let hasSeenPropSpread = false;
   while (!match(tt.slash) && !match(tt.jsxTagEnd) && !state.error) {
-    jsxParseAttribute();
+    if (eat(tt.braceL)) {
+      hasSeenPropSpread = true;
+      expect(tt.ellipsis);
+      parseMaybeAssign();
+      // }
+      nextJSXTagToken();
+      continue;
+    }
+    if (
+      hasSeenPropSpread &&
+      state.end - state.start === 3 &&
+      input.charCodeAt(state.start) === charCodes.lowercaseK &&
+      input.charCodeAt(state.start + 1) === charCodes.lowercaseE &&
+      input.charCodeAt(state.start + 2) === charCodes.lowercaseY
+    ) {
+      state.tokens[initialTokenIndex].jsxRole = JSXRole.KeyAfterPropSpread;
+    }
+    jsxParseNamespacedName(IdentifierRole.ObjectKey);
+    if (match(tt.eq)) {
+      nextJSXTagToken();
+      jsxParseAttributeValue();
+    }
   }
   const isSelfClosing = match(tt.slash);
   if (isSelfClosing) {
@@ -208,7 +225,10 @@ function jsxParseClosingElement(): void {
 // (starting after "<"), attributes, contents and closing tag.
 // Does not parse the last token.
 function jsxParseElementAt(): void {
-  const isSelfClosing = jsxParseOpeningElement();
+  const initialTokenIndex = state.tokens.length - 1;
+  state.tokens[initialTokenIndex].jsxRole = JSXRole.Normal;
+  let numExplicitChildren = 0;
+  const isSelfClosing = jsxParseOpeningElement(initialTokenIndex);
   if (!isSelfClosing) {
     nextJSXExprToken();
     while (true) {
@@ -218,13 +238,26 @@ function jsxParseElementAt(): void {
           if (match(tt.slash)) {
             nextJSXTagToken();
             jsxParseClosingElement();
+            if (
+              numExplicitChildren > 1 &&
+              // Key after prop spread takes precedence precedence over static children.
+              state.tokens[initialTokenIndex].jsxRole !== JSXRole.KeyAfterPropSpread
+            ) {
+              state.tokens[initialTokenIndex].jsxRole = JSXRole.StaticChildren;
+            }
             return;
           }
+          numExplicitChildren++;
           jsxParseElementAt();
           nextJSXExprToken();
           break;
 
         case tt.jsxText:
+          numExplicitChildren++;
+          nextJSXExprToken();
+          break;
+
+        case tt.jsxEmptyText:
           nextJSXExprToken();
           break;
 
@@ -233,8 +266,17 @@ function jsxParseElementAt(): void {
           if (match(tt.ellipsis)) {
             jsxParseSpreadChild();
             nextJSXExprToken();
+            // Spread children are a mechanism to explicitly mark children as
+            // static, so count it as 2 children to satisfy the "more than one
+            // child" condition.
+            numExplicitChildren += 2;
           } else {
-            jsxParseExpressionContainer();
+            // If we see {}, this is an empty pseudo-expression that doesn't
+            // count as a child.
+            if (!match(tt.braceR)) {
+              numExplicitChildren++;
+              parseExpression();
+            }
             nextJSXExprToken();
           }
 
diff --git a/src/parser/tokenizer/index.ts b/src/parser/tokenizer/index.ts
index b02acb3..08e6ccd 100644
--- a/src/parser/tokenizer/index.ts
+++ b/src/parser/tokenizer/index.ts
@@ -27,6 +27,19 @@ export enum IdentifierRole {
   ImportAccess,
 }
 
+/**
+ * Extra information on jsxTagStart tokens, used to determine which of the three
+ * jsx functions are called in the automatic transform.
+ */
+export enum JSXRole {
+  Normal,
+  // The element has at least two explicitly-specified children or has spread
+  // children.
+  StaticChildren,
+  // The element has a prop named "key" after a prop spread.
+  KeyAfterPropSpread,
+}
+
 export function isDeclaration(token: Token): boolean {
   const role = token.identifierRole;
   return (
@@ -97,6 +110,7 @@ export class Token {
     this.scopeDepth = state.scopeDepth;
     this.isType = state.isType;
     this.identifierRole = null;
+    this.jsxRole = null;
     this.shadowsGlobal = false;
     this.isAsyncOperation = false;
     this.contextId = null;
@@ -117,6 +131,7 @@ export class Token {
   scopeDepth: number;
   isType: boolean;
   identifierRole: IdentifierRole | null;
+  jsxRole: JSXRole | null;
   // Initially false for all tokens, then may be computed in a follow-up step that does scope
   // analysis.
   shadowsGlobal: boolean;
diff --git a/src/parser/tokenizer/types.ts b/src/parser/tokenizer/types.ts
index 36220b6..047fbf2 100644
--- a/src/parser/tokenizer/types.ts
+++ b/src/parser/tokenizer/types.ts
@@ -69,62 +69,63 @@ export enum TokenType {
   exponent = 54348, // ** prec:12 rightAssociative
   jsxName = 55296, // jsxName
   jsxText = 56320, // jsxText
-  jsxTagStart = 57856, // jsxTagStart startsExpr
-  jsxTagEnd = 58368, // jsxTagEnd
-  typeParameterStart = 59904, // typeParameterStart startsExpr
-  nonNullAssertion = 60416, // nonNullAssertion
-  _break = 61456, // break keyword
-  _case = 62480, // case keyword
-  _catch = 63504, // catch keyword
-  _continue = 64528, // continue keyword
-  _debugger = 65552, // debugger keyword
-  _default = 66576, // default keyword
-  _do = 67600, // do keyword
-  _else = 68624, // else keyword
-  _finally = 69648, // finally keyword
-  _for = 70672, // for keyword
-  _function = 72208, // function keyword startsExpr
-  _if = 72720, // if keyword
-  _return = 73744, // return keyword
-  _switch = 74768, // switch keyword
-  _throw = 76432, // throw keyword prefix startsExpr
-  _try = 76816, // try keyword
-  _var = 77840, // var keyword
-  _let = 78864, // let keyword
-  _const = 79888, // const keyword
-  _while = 80912, // while keyword
-  _with = 81936, // with keyword
-  _new = 83472, // new keyword startsExpr
-  _this = 84496, // this keyword startsExpr
-  _super = 85520, // super keyword startsExpr
-  _class = 86544, // class keyword startsExpr
-  _extends = 87056, // extends keyword
-  _export = 88080, // export keyword
-  _import = 89616, // import keyword startsExpr
-  _yield = 90640, // yield keyword startsExpr
-  _null = 91664, // null keyword startsExpr
-  _true = 92688, // true keyword startsExpr
-  _false = 93712, // false keyword startsExpr
-  _in = 94232, // in prec:8 keyword
-  _instanceof = 95256, // instanceof prec:8 keyword
-  _typeof = 96912, // typeof keyword prefix startsExpr
-  _void = 97936, // void keyword prefix startsExpr
-  _delete = 98960, // delete keyword prefix startsExpr
-  _async = 99856, // async keyword startsExpr
-  _get = 100880, // get keyword startsExpr
-  _set = 101904, // set keyword startsExpr
-  _declare = 102928, // declare keyword startsExpr
-  _readonly = 103952, // readonly keyword startsExpr
-  _abstract = 104976, // abstract keyword startsExpr
-  _static = 106000, // static keyword startsExpr
-  _public = 106512, // public keyword
-  _private = 107536, // private keyword
-  _protected = 108560, // protected keyword
-  _override = 109584, // override keyword
-  _as = 111120, // as keyword startsExpr
-  _enum = 112144, // enum keyword startsExpr
-  _type = 113168, // type keyword startsExpr
-  _implements = 114192, // implements keyword startsExpr
+  jsxEmptyText = 57344, // jsxEmptyText
+  jsxTagStart = 58880, // jsxTagStart startsExpr
+  jsxTagEnd = 59392, // jsxTagEnd
+  typeParameterStart = 60928, // typeParameterStart startsExpr
+  nonNullAssertion = 61440, // nonNullAssertion
+  _break = 62480, // break keyword
+  _case = 63504, // case keyword
+  _catch = 64528, // catch keyword
+  _continue = 65552, // continue keyword
+  _debugger = 66576, // debugger keyword
+  _default = 67600, // default keyword
+  _do = 68624, // do keyword
+  _else = 69648, // else keyword
+  _finally = 70672, // finally keyword
+  _for = 71696, // for keyword
+  _function = 73232, // function keyword startsExpr
+  _if = 73744, // if keyword
+  _return = 74768, // return keyword
+  _switch = 75792, // switch keyword
+  _throw = 77456, // throw keyword prefix startsExpr
+  _try = 77840, // try keyword
+  _var = 78864, // var keyword
+  _let = 79888, // let keyword
+  _const = 80912, // const keyword
+  _while = 81936, // while keyword
+  _with = 82960, // with keyword
+  _new = 84496, // new keyword startsExpr
+  _this = 85520, // this keyword startsExpr
+  _super = 86544, // super keyword startsExpr
+  _class = 87568, // class keyword startsExpr
+  _extends = 88080, // extends keyword
+  _export = 89104, // export keyword
+  _import = 90640, // import keyword startsExpr
+  _yield = 91664, // yield keyword startsExpr
+  _null = 92688, // null keyword startsExpr
+  _true = 93712, // true keyword startsExpr
+  _false = 94736, // false keyword startsExpr
+  _in = 95256, // in prec:8 keyword
+  _instanceof = 96280, // instanceof prec:8 keyword
+  _typeof = 97936, // typeof keyword prefix startsExpr
+  _void = 98960, // void keyword prefix startsExpr
+  _delete = 99984, // delete keyword prefix startsExpr
+  _async = 100880, // async keyword startsExpr
+  _get = 101904, // get keyword startsExpr
+  _set = 102928, // set keyword startsExpr
+  _declare = 103952, // declare keyword startsExpr
+  _readonly = 104976, // readonly keyword startsExpr
+  _abstract = 106000, // abstract keyword startsExpr
+  _static = 107024, // static keyword startsExpr
+  _public = 107536, // public keyword
+  _private = 108560, // private keyword
+  _protected = 109584, // protected keyword
+  _override = 110608, // override keyword
+  _as = 112144, // as keyword startsExpr
+  _enum = 113168, // enum keyword startsExpr
+  _type = 114192, // type keyword startsExpr
+  _implements = 115216, // implements keyword startsExpr
 }
 export function formatTokenType(tokenType: TokenType): string {
   switch (tokenType) {
@@ -240,6 +241,8 @@ export function formatTokenType(tokenType: TokenType): string {
       return "jsxName";
     case TokenType.jsxText:
       return "jsxText";
+    case TokenType.jsxEmptyText:
+      return "jsxEmptyText";
     case TokenType.jsxTagStart:
       return "jsxTagStart";
     case TokenType.jsxTagEnd:
diff --git a/src/parser/traverser/expression.ts b/src/parser/traverser/expression.ts
index 3981489..e90dece 100644
--- a/src/parser/traverser/expression.ts
+++ b/src/parser/traverser/expression.ts
@@ -448,7 +448,7 @@ export function parseExprAtom(): boolean {
     return false;
   }
 
-  if (match(tt.jsxText)) {
+  if (match(tt.jsxText) || match(tt.jsxEmptyText)) {
     parseLiteral();
     return false;
   } else if (match(tt.lessThan) && isJSXEnabled) {
diff --git a/src/parser/util/charcodes.ts b/src/parser/util/charcodes.ts
index 0253531..097d821 100644
--- a/src/parser/util/charcodes.ts
+++ b/src/parser/util/charcodes.ts
@@ -1,6 +1,7 @@
 export enum charCodes {
   backSpace = 8,
   lineFeed = 10, //  '\n'
+  tab = 9, //  '\t'
   carriageReturn = 13, //  '\r'
   shiftOut = 14,
   space = 32,
diff --git a/src/transformers/JSXTransformer.ts b/src/transformers/JSXTransformer.ts
index f13c002..cc0a101 100644
--- a/src/transformers/JSXTransformer.ts
+++ b/src/transformers/JSXTransformer.ts
@@ -181,7 +181,7 @@ export default class JSXTransformer extends Transformer {
         // Child JSX element
         this.tokens.appendCode(", ");
         this.processJSXTag();
-      } else if (this.tokens.matches1(tt.jsxText)) {
+      } else if (this.tokens.matches1(tt.jsxText) || this.tokens.matches1(tt.jsxEmptyText)) {
         this.processChildTextElement();
       } else {
         throw new Error("Unexpected token when processing JSX children.");


Test patch:
diff --git a/test/index-test.ts b/test/index-test.ts
index 71d5247..0003be8 100644
--- a/test/index-test.ts
+++ b/test/index-test.ts
@@ -13,21 +13,21 @@ if (foo) {
         {transforms: ["jsx", "imports"]},
       ),
       `\
-Location  Label  Raw            contextualKeyword scopeDepth isType identifierRole shadowsGlobal isAsyncOperation contextId rhsEndIndex isExpression numNullishCoalesceStarts numNullishCoalesceEnds isOptionalChainStart isOptionalChainEnd subscriptStartIndex nullishStartIndex
-1:1-1:3   if     if             0                 0                                                                                                  0                        0                                                                                                   
-1:4-1:5   (      (              0                 0                                                                                                  0                        0                                                                                                   
-1:5-1:8   name   foo            0                 0                 0                                                                                0                        0                                                                                                   
-1:8-1:9   )      )              0                 0                                                                                                  0                        0                                                                                                   
-1:10-1:11 {      {              0                 1                                                                                                  0                        0                                                                                                   
-2:3-2:10  name   console        0                 1                 0                                                                                0                        0                                                                                                   
-2:10-2:11 .      .              0                 1                                                                                                  0                        0                                                              5                                    
-2:11-2:14 name   log            0                 1                                                                                                  0                        0                                                                                                   
-2:14-2:15 (      (              0                 1                                                               1                                  0                        0                                                              5                                    
-2:15-2:29 string 'Hello world!' 0                 1                                                                                                  0                        0                                                                                                   
-2:29-2:30 )      )              0                 1                                                               1                                  0                        0                                                                                                   
-2:30-2:31 ;      ;              0                 1                                                                                                  0                        0                                                                                                   
-3:1-3:2   }      }              0                 1                                                                                                  0                        0                                                                                                   
-3:2-3:2   eof                   0                 0                                                                                                  0                        0                                                                                                   `,
+Location  Label  Raw            contextualKeyword scopeDepth isType identifierRole jsxRole shadowsGlobal isAsyncOperation contextId rhsEndIndex isExpression numNullishCoalesceStarts numNullishCoalesceEnds isOptionalChainStart isOptionalChainEnd subscriptStartIndex nullishStartIndex
+1:1-1:3   if     if             0                 0                                                                                                          0                        0                                                                                                   
+1:4-1:5   (      (              0                 0                                                                                                          0                        0                                                                                                   
+1:5-1:8   name   foo            0                 0                 0                                                                                        0                        0                                                                                                   
+1:8-1:9   )      )              0                 0                                                                                                          0                        0                                                                                                   
+1:10-1:11 {      {              0                 1                                                                                                          0                        0                                                                                                   
+2:3-2:10  name   console        0                 1                 0                                                                                        0                        0                                                                                                   
+2:10-2:11 .      .              0                 1                                                                                                          0                        0                                                              5                                    
+2:11-2:14 name   log            0                 1                                                                                                          0                        0                                                                                                   
+2:14-2:15 (      (              0                 1                                                                       1                                  0                        0                                                              5                                    
+2:15-2:29 string 'Hello world!' 0                 1                                                                                                          0                        0                                                                                                   
+2:29-2:30 )      )              0                 1                                                                       1                                  0                        0                                                                                                   
+2:30-2:31 ;      ;              0                 1                                                                                                          0                        0                                                                                                   
+3:1-3:2   }      }              0                 1                                                                                                          0                        0                                                                                                   
+3:2-3:2   eof                   0                 0                                                                                                          0                        0                                                                                                   `,
     );
   });
 });
diff --git a/test/tokens-test.ts b/test/tokens-test.ts
index 57bc8bb..448fa9f 100644
--- a/test/tokens-test.ts
+++ b/test/tokens-test.ts
@@ -1,9 +1,9 @@
 import * as assert from "assert";
 
 import {parse} from "../src/parser";
-import {IdentifierRole, Token} from "../src/parser/tokenizer";
+import {IdentifierRole, JSXRole, Token} from "../src/parser/tokenizer";
 import {ContextualKeyword} from "../src/parser/tokenizer/keywords";
-import {TokenType as tt} from "../src/parser/tokenizer/types";
+import {TokenType, TokenType as tt} from "../src/parser/tokenizer/types";
 
 type SimpleToken = Token & {label?: string};
 type TokenExpectation = {[K in keyof SimpleToken]?: SimpleToken[K]};
@@ -29,6 +29,13 @@ function assertTokens(
   assert.deepStrictEqual(projectedTokens, expectedTokens, helpMessage);
 }
 
+function assertFirstJSXRole(code: string, expectedJSXRole: JSXRole): void {
+  const tokens = parse(code, true, true, false).tokens;
+  const jsxTagStartTokens = tokens.filter((t) => t.type === TokenType.jsxTagStart);
+  assert.ok(jsxTagStartTokens.length > 0);
+  assert.strictEqual(jsxTagStartTokens[0].jsxRole, expectedJSXRole);
+}
+
 describe("tokens", () => {
   it("properly provides identifier roles for const, let, and var", () => {
     assertTokens(
@@ -224,7 +231,7 @@ describe("tokens", () => {
         {type: tt.braceR},
         {type: tt.slash},
         {type: tt.jsxTagEnd},
-        {type: tt.jsxText},
+        {type: tt.jsxEmptyText},
         {type: tt.jsxTagStart},
         {type: tt.slash},
         {type: tt.jsxName},
@@ -498,4 +505,115 @@ describe("tokens", () => {
       {isFlow: true},
     );
   });
+
+  it("treats single-child JSX as non-static", () => {
+    assertFirstJSXRole("const elem = <div>Hello</div>;", JSXRole.Normal);
+  });
+
+  it("treats self-closing JSX as non-static", () => {
+    assertFirstJSXRole("const elem = <div />;", JSXRole.Normal);
+  });
+
+  it("treats JSX with two element children as static", () => {
+    assertFirstJSXRole("const elem = <div><span /><span /></div>;", JSXRole.StaticChildren);
+  });
+
+  it("treats JSX with two expression children as static", () => {
+    assertFirstJSXRole("const elem = <div>{a}{b}</div>;", JSXRole.StaticChildren);
+  });
+
+  it("treats JSX with two whitespace-only children as static", () => {
+    assertFirstJSXRole("const elem = <div> {} </div>;", JSXRole.StaticChildren);
+  });
+
+  it("treats JSX with non-spread children as non-static", () => {
+    assertFirstJSXRole("const elem = <div>{elements}</div>;", JSXRole.Normal);
+  });
+
+  it("treats JSX with spread children as static", () => {
+    assertFirstJSXRole("const elem = <div>{...elements}</div>;", JSXRole.StaticChildren);
+  });
+
+  it("treats JSX with one empty and one space-only text as non-static", () => {
+    assertFirstJSXRole(
+      `
+      const elem = <div> {} 
+      </div>;
+    `,
+      JSXRole.Normal,
+    );
+  });
+
+  it("treats JSX with two empty text tokens as non-static", () => {
+    assertFirstJSXRole(
+      `
+      const elem = (
+        <div>
+          {} 
+        </div>
+      );
+    `,
+      JSXRole.Normal,
+    );
+  });
+
+  it("treats JSX with one space-only text and one text with contents as static", () => {
+    assertFirstJSXRole(
+      `
+      const elem = (
+        <div>    {}
+          
+          a
+          
+        </div>
+      );
+    `,
+      JSXRole.StaticChildren,
+    );
+  });
+
+  it("detects key after prop spread", () => {
+    assertFirstJSXRole("const elem = <div {...props} key={1} />;", JSXRole.KeyAfterPropSpread);
+  });
+
+  it("does not mark as key-after-prop-spread if there is just a key", () => {
+    assertFirstJSXRole("const elem = <div key={1} />;", JSXRole.Normal);
+  });
+
+  it("does not mark as key-after-prop-spread the key is after regular props", () => {
+    assertFirstJSXRole("const elem = <div foo='a' bar='b' key={1} />;", JSXRole.Normal);
+  });
+
+  it("does not mark as key-after-prop-spread if the prop spread is afterward", () => {
+    assertFirstJSXRole("const elem = <div key={1} {...props} />;", JSXRole.Normal);
+  });
+
+  it("marks as key-after-prop-spread if there is a prop spread before and after", () => {
+    assertFirstJSXRole(
+      "const elem = <div {...props} key={1} {...props2} />;",
+      JSXRole.KeyAfterPropSpread,
+    );
+  });
+
+  it("marks as key-after-prop-spread if there are two keys", () => {
+    assertFirstJSXRole(
+      "const elem = <div key={1} {...props} key={2} />;",
+      JSXRole.KeyAfterPropSpread,
+    );
+  });
+
+  it("does not mark as key-after-prop-spread for camelCase props starting with key", () => {
+    assertFirstJSXRole("const elem = <div {...props} keyLimePie={1} />;", JSXRole.Normal);
+  });
+
+  it("does not mark as key-after-prop-spread for hyphenated props starting with key", () => {
+    assertFirstJSXRole("const elem = <div {...props} key-lime-pie={1} />;", JSXRole.Normal);
+  });
+
+  it("marks key-after-prop-spread with higher precedence than static children", () => {
+    assertFirstJSXRole(
+      "const elem = <div {...props} key={1}>{a}{b}</div>;",
+      JSXRole.KeyAfterPropSpread,
+    );
+  });
 });