Repository: eemeli/yaml
Pull request #249 (2021-03-28 18:21:53)

Split flow collections into items in Parser
The shape of FlowCollection `items` changes from `Token[]` to `{ start, key, sep, value }[]`, so that it now matches the `items` of BlockMap and BlockSequence. This means that the corresponding composer is completely rewritten.

Ping @evidolob, @gorkem, @JPinkney


Patch:
diff --git a/docs/07_parsing_yaml.md b/docs/07_parsing_yaml.md
index b8df5a2..2176eb5 100644
--- a/docs/07_parsing_yaml.md
+++ b/docs/07_parsing_yaml.md
@@ -187,10 +187,17 @@ Some of the most common node properties include:
 | `offset`              | `number`        | The start index within the source string or character stream.                                                                                 |
 | `source`              | `string`        | A raw string representation of the node's value, including all newlines and indentation.                                                      |
 | `indent`              | `number`        | The indent level of the current line; mostly just for internal use.                                                                           |
-| `items`               | `{ ... }[]`     | The contents of a collection; shape depends on the collection type, and may include `key: Token` and `value: Token`.                          |
+| `items`               | `Item[]`        | The contents of a collection; exact shape depends on the collection type.                                                                     |
 | `start`, `sep`, `end` | `SourceToken[]` | Content before, within, and after "actual" values. Includes item and collection indicators, anchors, tags, comments, as well as other things. |
 
-As an implementation detail, block and flow collections are parsed and presented rather differently due to their structural differences.
+Collection items contain some subset of the following properties:
+
+| Item property | Type            | Description                                                                                                                |
+| ------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------- |
+| `start`       | `SourceToken[]` | Always defined. Content before the actual value. May include comments that are later assigned to the preceding item.       |
+| `key`         | `Token ⎮ null`  | Set for key/value pairs only, so never used in block sequences.                                                            |
+| `sep`         | `SourceToken[]` | Content between the key and the value. If defined, indicates that the `key` logically exists, even if its value is `null`. |
+| `value`       | `Token ⎮ null`  | The value. Normally set, but may be left out for e.g. explicit keys with no matching value.                                |
 
 ### Counting Lines
 
diff --git a/docs/08_errors.md b/docs/08_errors.md
index 0a67920..28e9592 100644
--- a/docs/08_errors.md
+++ b/docs/08_errors.md
@@ -28,6 +28,7 @@ To identify errors for special handling, you should primarily use `code` to diff
 | `BAD_DIRECTIVE`          | Only the `%YAML` and `%TAG` directives are supported, and they need to follow the specified strucutre.                                                                       |
 | `BAD_DQ_ESCAPE`          | Double-quotes strings may include `\` escaped content, but that needs to be valid.                                                                                           |
 | `BAD_INDENT`             | Indentation is important in YAML, and collection items need to all start at the same level. Block scalars are also picky about their leading content.                        |
+| `BAD_PROP_ORDER`         | Anchors and tags must be placed after the `?`, `:` and `-` indicators.                                                                                                       |
 | `BAD_SCALAR_START`       | Plain scalars cannot start with a block scalar indicator, or one of the two reserved characters: `@` and <code>`</code>. To fix, use a block or quoted scalar for the value. |
 | `BLOCK_AS_IMPLICIT_KEY`  | There's probably something wrong with the indentation, or you're trying to parse something like `a: b: c`, where it's not clear what's the key and what's the value.         |
 | `BLOCK_IN_FLOW`          | YAML scalars and collections both have block and flow styles. Flow is allowed within block, but not the other way around.                                                    |
@@ -40,7 +41,6 @@ To identify errors for special handling, you should primarily use `code` to diff
 | `MULTIPLE_ANCHORS`       | A node is only allowed to have one anchor.                                                                                                                                   |
 | `MULTIPLE_DOCS`          | A YAML stream may include multiple documents. If yours does, you'll need to use `parseAllDocuments()` to work with it.                                                       |
 | `MULTIPLE_TAGS`          | A node is only allowed to have one tag.                                                                                                                                      |
-| `PROP_BEFORE_SEP`        | For an explicit key, anchors and tags must be after the `?` indicator                                                                                                        |
 | `TAB_AS_INDENT`          | Only spaces are allowed as indentation.                                                                                                                                      |
 | `TAG_RESOLVE_FAILED`     | Something went wrong when resolving a node's tag with the current schema.                                                                                                    |
 | `UNEXPECTED_TOKEN`       | A token was encountered in a place where it wasn't expected.                                                                                                                 |
diff --git a/src/compose/resolve-flow-collection.ts b/src/compose/resolve-flow-collection.ts
index bd142de..aadaa29 100644
--- a/src/compose/resolve-flow-collection.ts
+++ b/src/compose/resolve-flow-collection.ts
@@ -1,11 +1,12 @@
-import { isNode, isPair, ParsedNode } from '../nodes/Node.js'
+import { isPair } from '../nodes/Node.js'
 import { Pair } from '../nodes/Pair.js'
 import { YAMLMap } from '../nodes/YAMLMap.js'
 import { YAMLSeq } from '../nodes/YAMLSeq.js'
-import type { FlowCollection, SourceToken, Token } from '../parse/tokens.js'
+import type { FlowCollection } from '../parse/tokens.js'
 import type { ComposeContext, ComposeNode } from './compose-node.js'
 import type { ComposeErrorHandler } from './composer.js'
 import { resolveEnd } from './resolve-end.js'
+import { resolveProps } from './resolve-props.js'
 import { containsNewline } from './util-contains-newline.js'
 
 export function resolveFlowCollection(
@@ -15,244 +16,198 @@ export function resolveFlowCollection(
   onError: ComposeErrorHandler
 ) {
   const isMap = fc.start.source === '{'
-  const coll = isMap ? new YAMLMap(ctx.schema) : new YAMLSeq(ctx.schema)
+  const fcName = isMap ? 'flow map' : 'flow sequence'
+  const coll = isMap
+    ? (new YAMLMap(ctx.schema) as YAMLMap.Parsed)
+    : (new YAMLSeq(ctx.schema) as YAMLSeq.Parsed)
   coll.flow = true
 
-  let key: ParsedNode | null = null
-  let value: ParsedNode | null = null
-
-  let spaceBefore = false
-  let comment = ''
-  let hasSpace = false
-  let newlines = ''
-  let anchor = ''
-  let tagName = ''
-
-  let offset = fc.offset + 1
-  let atLineStart = false
-  let atExplicitKey = false
-  let atValueEnd = false
-  let nlAfterValueInSeq = false
-  let seqKeyToken: Token | null = null
-
-  function getProps() {
-    const props = { spaceBefore, comment, anchor, tagName }
-
-    spaceBefore = false
-    comment = ''
-    newlines = ''
-    anchor = ''
-    tagName = ''
-
-    return props
-  }
-
-  function addItem(pos: number) {
-    if (value) {
-      if (comment) value.comment = comment
-    } else {
-      value = composeEmptyNode(ctx, offset, fc.items, pos, getProps(), onError)
-    }
-    if (isMap || atExplicitKey) {
-      coll.items.push(key ? new Pair(key, value) : new Pair(value))
-    } else {
-      const seq = coll as YAMLSeq
-      if (key) {
-        const map = new YAMLMap(ctx.schema)
-        map.flow = true
-        map.items.push(new Pair(key, value))
-        seq.items.push(map)
-      } else seq.items.push(value)
-    }
-  }
-
+  let offset = fc.offset
   for (let i = 0; i < fc.items.length; ++i) {
-    const token = fc.items[i]
-    let isSourceToken = true
-    switch (token.type) {
-      case 'space':
-        hasSpace = true
-        break
-      case 'comment': {
-        if (ctx.options.strict && !hasSpace)
+    const { start, key, sep, value } = fc.items[i]
+    const props = resolveProps(start, {
+      ctx,
+      flow: fcName,
+      indicator: 'explicit-key-ind',
+      offset,
+      onError,
+      startOnNewline: false
+    })
+    if (!props.found) {
+      if (!props.anchor && !props.tagName && !sep && !value) {
+        if (i === 0 && props.comma)
           onError(
-            offset,
-            'COMMENT_SPACE',
-            'Comments must be separated from other tokens by white space characters'
+            props.comma.offset,
+            'UNEXPECTED_TOKEN',
+            `Unexpected , in ${fcName}`
           )
-        const cb = token.source.substring(1)
-        if (!comment) comment = cb
-        else comment += newlines + cb
-        atLineStart = false
-        newlines = ''
-        break
-      }
-      case 'newline':
-        if (atLineStart && !comment) spaceBefore = true
-        if (atValueEnd) {
-          if (comment) {
-            let node = coll.items[coll.items.length - 1]
-            if (isPair(node)) node = node.value || node.key
-            /* istanbul ignore else should not happen */
-            if (isNode(node)) node.comment = comment
-            else
-              onError(
-                offset,
-                'IMPOSSIBLE',
-                'Error adding trailing comment to node'
-              )
-            comment = ''
-          }
-          atValueEnd = false
-        } else {
-          newlines += token.source
-          if (!isMap && !key && value) nlAfterValueInSeq = true
-        }
-        atLineStart = true
-        hasSpace = true
-        break
-      case 'anchor':
-        if (anchor)
+        else if (i < fc.items.length - 1)
           onError(
-            offset,
-            'MULTIPLE_ANCHORS',
-            'A node can have at most one anchor'
+            props.start,
+            'UNEXPECTED_TOKEN',
+            `Unexpected empty item in ${fcName}`
           )
-        anchor = token.source.substring(1)
-        atLineStart = false
-        atValueEnd = false
-        hasSpace = false
-        break
-      case 'tag': {
-        if (tagName)
-          onError(offset, 'MULTIPLE_TAGS', 'A node can have at most one tag')
-        const tn = ctx.directives.tagName(token.source, m =>
-          onError(offset, 'TAG_RESOLVE_FAILED', m)
-        )
-        if (tn) tagName = tn
-        atLineStart = false
-        atValueEnd = false
-        hasSpace = false
-        break
+        if (props.comment) {
+          if (coll.comment) coll.comment += '\n' + props.comment
+          else coll.comment = props.comment
+        }
+        continue
       }
-      case 'explicit-key-ind':
-        if (anchor || tagName)
-          onError(
-            offset,
-            'PROP_BEFORE_SEP',
-            'Anchors and tags must be after the ? indicator'
-          )
-        atExplicitKey = true
-        atLineStart = false
-        atValueEnd = false
-        hasSpace = false
-        break
-      case 'map-value-ind': {
-        if (key) {
-          if (value) {
-            onError(
-              offset,
-              'BLOCK_AS_IMPLICIT_KEY',
-              'Missing {} around pair used as mapping key'
-            )
-            const map = new YAMLMap(ctx.schema)
-            map.flow = true
-            map.items.push(new Pair(key, value))
-            map.range = [key.range[0], value.range[1]]
-            key = map as YAMLMap.Parsed
-            value = null
-          } // else explicit key
-        } else if (value) {
-          if (ctx.options.strict) {
-            const slMsg =
-              'Implicit keys of flow sequence pairs need to be on a single line'
-            if (nlAfterValueInSeq)
-              onError(offset, 'MULTILINE_IMPLICIT_KEY', slMsg)
-            else if (seqKeyToken) {
-              if (containsNewline(seqKeyToken))
-                onError(offset, 'MULTILINE_IMPLICIT_KEY', slMsg)
-              if (seqKeyToken.offset < offset - 1024)
-                onError(
-                  offset,
-                  'KEY_OVER_1024_CHARS',
-                  'The : indicator must be at most 1024 chars after the start of an implicit flow sequence key'
-                )
-              seqKeyToken = null
-            }
+      if (!isMap && ctx.options.strict && containsNewline(key))
+        onError(
+          props.start,
+          'MULTILINE_IMPLICIT_KEY',
+          'Implicit keys of flow sequence pairs need to be on a single line'
+        )
+    }
+    if (i === 0) {
+      if (props.comma)
+        onError(
+          props.comma.offset,
+          'UNEXPECTED_TOKEN',
+          `Unexpected , in ${fcName}`
+        )
+    } else {
+      if (!props.comma)
+        onError(
+          props.start,
+          'MISSING_CHAR',
+          `Missing , between ${fcName} items`
+        )
+      if (props.comment) {
+        let prevItemComment = ''
+        loop: for (const st of start) {
+          switch (st.type) {
+            case 'comma':
+            case 'space':
+              break
+            case 'comment':
+              prevItemComment = st.source.substring(1)
+              break loop
+            default:
+              break loop
           }
-          key = value
-          value = null
-        } else {
-          key = composeEmptyNode(ctx, offset, fc.items, i, getProps(), onError)
         }
-        if (comment) {
-          key.comment = comment
-          comment = ''
+        if (prevItemComment) {
+          let prev = coll.items[coll.items.length - 1]
+          if (isPair(prev)) prev = prev.value || prev.key
+          if (prev.comment) prev.comment += '\n' + prevItemComment
+          else prev.comment = prevItemComment
+          props.comment = props.comment.substring(prevItemComment.length + 1)
         }
-        atExplicitKey = false
-        atValueEnd = false
-        hasSpace = false
-        break
       }
-      case 'comma':
-        if (key || value || anchor || tagName || atExplicitKey) addItem(i)
-        else
-          onError(
-            offset,
-            'UNEXPECTED_TOKEN',
-            `Unexpected , in flow ${isMap ? 'map' : 'sequence'}`
-          )
-        key = null
-        value = null
-        atExplicitKey = false
-        atValueEnd = true
-        hasSpace = false
-        nlAfterValueInSeq = false
-        seqKeyToken = null
-        break
-      case 'block-map':
-      case 'block-seq':
+    }
+
+    for (const token of [key, value])
+      if (token && (token.type === 'block-map' || token.type === 'block-seq'))
         onError(
-          offset,
+          token.offset,
           'BLOCK_IN_FLOW',
           'Block collections are not allowed within flow collections'
         )
-      // fallthrough
-      default: {
-        if (value)
+
+    if (!isMap && !sep && !props.found) {
+      // item is a value in a seq
+      // → key & sep are empty, start does not include ? or :
+      const valueNode = value
+        ? composeNode(ctx, value, props, onError)
+        : composeEmptyNode(ctx, props.end, sep, null, props, onError)
+      ;(coll as YAMLSeq).items.push(valueNode)
+      offset = valueNode.range[1]
+    } else {
+      // item is a key+value pair
+
+      // key value
+      const keyStart = props.end
+      const keyNode = key
+        ? composeNode(ctx, key, props, onError)
+        : composeEmptyNode(ctx, keyStart, start, null, props, onError)
+
+      // value properties
+      const valueProps = resolveProps(sep || [], {
+        ctx,
+        flow: fcName,
+        indicator: 'map-value-ind',
+        offset: keyNode.range[1],
+        onError,
+        startOnNewline: false
+      })
+
+      if (valueProps.found) {
+        if (!isMap && !props.found && ctx.options.strict) {
+          if (sep)
+            for (const st of sep) {
+              if (st === valueProps.found) break
+              if (st.type === 'newline') {
+                onError(
+                  st.offset,
+                  'MULTILINE_IMPLICIT_KEY',
+                  'Implicit keys of flow sequence pairs need to be on a single line'
+                )
+                break
+              }
+            }
+          if (props.start < valueProps.found.offset - 1024)
+            onError(
+              valueProps.found.offset,
+              'KEY_OVER_1024_CHARS',
+              'The : indicator must be at most 1024 chars after the start of an implicit flow sequence key'
+            )
+        }
+      } else if (value) {
+        if ('source' in value && value.source && value.source[0] === ':')
+          onError(
+            value.offset,
+            'MISSING_CHAR',
+            `Missing space after : in ${fcName}`
+          )
+        else
           onError(
-            offset,
+            valueProps.start,
             'MISSING_CHAR',
-            'Missing , between flow collection items'
+            `Missing , or : between ${fcName} items`
           )
-        if (!isMap && !key && !atExplicitKey) seqKeyToken = token
-        value = composeNode(ctx, token, getProps(), onError)
-        offset = value.range[1]
-        atLineStart = false
-        isSourceToken = false
-        atValueEnd = false
-        hasSpace = false
       }
+
+      // value value
+      const valueNode = value
+        ? composeNode(ctx, value, valueProps, onError)
+        : valueProps.found
+        ? composeEmptyNode(ctx, valueProps.end, sep, null, valueProps, onError)
+        : null
+      if (!valueNode && valueProps.comment) {
+        if (keyNode.comment) keyNode.comment += '\n' + valueProps.comment
+        else keyNode.comment = valueProps.comment
+      }
+
+      const pair = new Pair(keyNode, valueNode)
+      if (isMap) (coll as YAMLMap.Parsed).items.push(pair)
+      else {
+        const map = new YAMLMap(ctx.schema)
+        map.flow = true
+        map.items.push(pair)
+        ;(coll as YAMLSeq).items.push(map)
+      }
+      offset = valueNode ? valueNode.range[1] : valueProps.end
     }
-    if (isSourceToken) offset += (token as SourceToken).source.length
   }
-  if (key || value || anchor || tagName || atExplicitKey)
-    addItem(fc.items.length)
 
   const expectedEnd = isMap ? '}' : ']'
   const [ce, ...ee] = fc.end
   if (!ce || ce.source !== expectedEnd) {
-    const cs = isMap ? 'map' : 'sequence'
     onError(
-      offset,
+      offset + 1,
       'MISSING_CHAR',
-      `Expected flow ${cs} to end with ${expectedEnd}`
+      `Expected ${fcName} to end with ${expectedEnd}`
     )
   }
   if (ce) offset += ce.source.length
   if (ee.length > 0) {
     const end = resolveEnd(ee, offset, ctx.options.strict, onError)
-    if (end.comment) coll.comment = comment
+    if (end.comment) {
+      if (coll.comment) coll.comment += '\n' + end.comment
+      else coll.comment = end.comment
+    }
     offset = end.offset
   }
 
diff --git a/src/compose/resolve-props.ts b/src/compose/resolve-props.ts
index 5eb3444..419c27b 100644
--- a/src/compose/resolve-props.ts
+++ b/src/compose/resolve-props.ts
@@ -4,6 +4,7 @@ import type { ComposeErrorHandler } from './composer.js'
 
 export interface ResolvePropsArg {
   ctx: ComposeContext
+  flow?: string
   indicator: 'doc-start' | 'explicit-key-ind' | 'map-value-ind' | 'seq-item-ind'
   offset: number
   onError: ComposeErrorHandler
@@ -12,7 +13,7 @@ export interface ResolvePropsArg {
 
 export function resolveProps(
   tokens: SourceToken[],
-  { ctx, indicator, offset, onError, startOnNewline }: ResolvePropsArg
+  { ctx, flow, indicator, offset, onError, startOnNewline }: ResolvePropsArg
 ) {
   let spaceBefore = false
   let atNewline = startOnNewline
@@ -22,14 +23,21 @@ export function resolveProps(
   let hasNewline = false
   let anchor = ''
   let tagName = ''
+  let comma: SourceToken | null = null
   let found: SourceToken | null = null
   let start: number | null = null
   for (const token of tokens) {
     switch (token.type) {
       case 'space':
-        // At the doc level, tabs at line start may be parsed as leading
-        // white space rather than indentation.
-        if (atNewline && indicator !== 'doc-start' && token.source[0] === '\t')
+        // At the doc level, tabs at line start may be parsed
+        // as leading white space rather than indentation.
+        // In a flow collection, only the parser handles indent.
+        if (
+          !flow &&
+          atNewline &&
+          indicator !== 'doc-start' &&
+          token.source[0] === '\t'
+        )
           onError(
             token.offset,
             'TAB_AS_INDENT',
@@ -87,10 +95,26 @@ export function resolveProps(
       }
       case indicator:
         // Could here handle preceding comments differently
+        if (anchor || tagName)
+          onError(
+            token.offset,
+            'BAD_PROP_ORDER',
+            `Anchors and tags must be after the ${token.source} indicator`
+          )
         found = token
         atNewline = false
         hasSpace = false
         break
+      case 'comma':
+        if (flow) {
+          if (comma)
+            onError(token.offset, 'UNEXPECTED_TOKEN', `Unexpected , in ${flow}`)
+          comma = token
+          atNewline = false
+          hasSpace = false
+          break
+        }
+      // else fallthrough
       default:
         onError(
           token.offset,
@@ -104,6 +128,7 @@ export function resolveProps(
   const last = tokens[tokens.length - 1]
   const end = last ? last.offset + last.source.length : offset
   return {
+    comma,
     found,
     spaceBefore,
     comment,
diff --git a/src/compose/util-contains-newline.ts b/src/compose/util-contains-newline.ts
index cd0c26a..32cd8d3 100644
--- a/src/compose/util-contains-newline.ts
+++ b/src/compose/util-contains-newline.ts
@@ -7,20 +7,16 @@ export function containsNewline(key: Token | null | undefined) {
     case 'scalar':
     case 'double-quoted-scalar':
     case 'single-quoted-scalar':
-      return key.source.includes('\n')
+      if (key.source.includes('\n')) return true
+      if (key.end)
+        for (const st of key.end) if (st.type === 'newline') return true
+      return false
     case 'flow-collection':
-      for (const token of key.items) {
-        switch (token.type) {
-          case 'newline':
-            return true
-          case 'alias':
-          case 'scalar':
-          case 'double-quoted-scalar':
-          case 'single-quoted-scalar':
-          case 'flow-collection':
-            if (containsNewline(token)) return true
-            break
-        }
+      for (const it of key.items) {
+        for (const st of it.start) if (st.type === 'newline') return true
+        if (it.sep)
+          for (const st of it.sep) if (st.type === 'newline') return true
+        if (containsNewline(it.key) || containsNewline(it.value)) return true
       }
       return false
     default:
diff --git a/src/errors.ts b/src/errors.ts
index 5bc3a7d..97d3496 100644
--- a/src/errors.ts
+++ b/src/errors.ts
@@ -5,6 +5,7 @@ export type ErrorCode =
   | 'BAD_DIRECTIVE'
   | 'BAD_DQ_ESCAPE'
   | 'BAD_INDENT'
+  | 'BAD_PROP_ORDER'
   | 'BAD_SCALAR_START'
   | 'BLOCK_AS_IMPLICIT_KEY'
   | 'BLOCK_IN_FLOW'
@@ -17,7 +18,6 @@ export type ErrorCode =
   | 'MULTIPLE_ANCHORS'
   | 'MULTIPLE_DOCS'
   | 'MULTIPLE_TAGS'
-  | 'PROP_BEFORE_SEP'
   | 'TAB_AS_INDENT'
   | 'TAG_RESOLVE_FAILED'
   | 'UNEXPECTED_TOKEN'
diff --git a/src/parse/parser.ts b/src/parse/parser.ts
index e770dc1..b74220e 100644
--- a/src/parse/parser.ts
+++ b/src/parse/parser.ts
@@ -66,7 +66,7 @@ function atFirstEmptyLineAfterComments(start: SourceToken[]) {
 }
 
 function isFlowToken(
-  token: Token | null
+  token: Token | null | undefined
 ): token is FlowScalar | FlowCollection {
   switch (token?.type) {
     case 'alias':
@@ -116,6 +116,27 @@ function getFirstKeyStartProps(prev: SourceToken[]) {
   return prev.splice(i, prev.length)
 }
 
+function fixFlowSeqItems(fc: FlowCollection) {
+  if (fc.start.type === 'flow-seq-start') {
+    for (const it of fc.items) {
+      if (
+        it.sep &&
+        !it.value &&
+        !includesToken(it.start, 'explicit-key-ind') &&
+        !includesToken(it.sep, 'map-value-ind')
+      ) {
+        if (it.key) it.value = it.key
+        delete it.key
+        if (isFlowToken(it.value)) {
+          if (it.value.end) Array.prototype.push.apply(it.value.end, it.sep)
+          else it.value.end = it.sep
+        } else Array.prototype.push.apply(it.start, it.sep)
+        delete it.sep
+      }
+    }
+  }
+}
+
 /**
  * A YAML concrete syntax tree (CST) parser
  *
@@ -309,6 +330,7 @@ export class Parser {
       // For these, parent indent is needed instead of own
       if (token.type === 'block-scalar' || token.type === 'flow-collection')
         token.indent = 'indent' in top ? top.indent : -1
+      if (token.type === 'flow-collection') fixFlowSeqItems(token)
       switch (top.type) {
         case 'document':
           top.value = token
@@ -337,9 +359,14 @@ export class Parser {
           else it.value = token
           break
         }
-        case 'flow-collection':
-          top.items.push(token)
-          break
+        case 'flow-collection': {
+          const it = top.items[top.items.length - 1]
+          if (!it || it.value)
+            top.items.push({ start: [], key: token, sep: [] })
+          else if (it.sep) it.value = token
+          else Object.assign(it, { key: token, sep: [] })
+          return
+        }
         /* istanbul ignore next should not happen */
         default:
           this.pop()
@@ -652,31 +679,48 @@ export class Parser {
   }
 
   private flowCollection(fc: FlowCollection) {
+    const it = fc.items[fc.items.length - 1]
     if (this.type === 'flow-error-end') {
-      let top
+      let top: Token | undefined
       do {
         this.pop()
         top = this.peek(1)
       } while (top && top.type === 'flow-collection')
     } else if (fc.end.length === 0) {
       switch (this.type) {
-        case 'space':
-        case 'comment':
-        case 'newline':
         case 'comma':
         case 'explicit-key-ind':
+          if (!it || it.sep) fc.items.push({ start: [this.sourceToken] })
+          else it.start.push(this.sourceToken)
+          return
+
         case 'map-value-ind':
+          if (!it || it.value)
+            fc.items.push({ start: [], key: null, sep: [this.sourceToken] })
+          else if (it.sep) it.sep.push(this.sourceToken)
+          else Object.assign(it, { key: null, sep: [this.sourceToken] })
+          return
+
+        case 'space':
+        case 'comment':
+        case 'newline':
         case 'anchor':
         case 'tag':
-          fc.items.push(this.sourceToken)
+          if (!it || it.value) fc.items.push({ start: [this.sourceToken] })
+          else if (it.sep) it.sep.push(this.sourceToken)
+          else it.start.push(this.sourceToken)
           return
 
         case 'alias':
         case 'scalar':
         case 'single-quoted-scalar':
-        case 'double-quoted-scalar':
-          fc.items.push(this.flowScalar(this.type))
+        case 'double-quoted-scalar': {
+          const fs = this.flowScalar(this.type)
+          if (!it || it.value) fc.items.push({ start: [], key: fs, sep: [] })
+          else if (it.sep) this.stack.push(fs)
+          else Object.assign(it, { key: fs, sep: [] })
           return
+        }
 
         case 'flow-map-end':
         case 'flow-seq-end':
@@ -706,6 +750,7 @@ export class Parser {
       ) {
         const prev = getPrevProps(parent)
         const start = getFirstKeyStartProps(prev)
+        fixFlowSeqItems(fc)
         const sep = fc.end.splice(1, fc.end.length)
         sep.push(this.sourceToken)
         const map: BlockMap = {
@@ -771,14 +816,18 @@ export class Parser {
           indent: this.indent,
           items: [{ start: [this.sourceToken] }]
         } as BlockSequence
-      case 'explicit-key-ind':
+      case 'explicit-key-ind': {
         this.onKeyLine = true
+        const prev = getPrevProps(parent)
+        const start = getFirstKeyStartProps(prev)
+        start.push(this.sourceToken)
         return {
           type: 'block-map',
           offset: this.offset,
           indent: this.indent,
-          items: [{ start: [this.sourceToken] }]
+          items: [{ start }]
         } as BlockMap
+      }
       case 'map-value-ind': {
         this.onKeyLine = true
         const prev = getPrevProps(parent)
diff --git a/src/parse/tokens.ts b/src/parse/tokens.ts
index 2703f78..8ee1993 100644
--- a/src/parse/tokens.ts
+++ b/src/parse/tokens.ts
@@ -87,7 +87,12 @@ export interface BlockSequence {
   type: 'block-seq'
   offset: number
   indent: number
-  items: Array<{ start: SourceToken[]; value?: Token; sep?: never }>
+  items: Array<{
+    start: SourceToken[]
+    key?: never
+    sep?: never
+    value?: Token
+  }>
 }
 
 export interface FlowCollection {
@@ -95,7 +100,12 @@ export interface FlowCollection {
   offset: number
   indent: number
   start: SourceToken
-  items: Array<Token>
+  items: Array<{
+    start: SourceToken[]
+    key?: Token | null
+    sep?: SourceToken[]
+    value?: Token
+  }>
   end: SourceToken[]
 }
 


Test patch:
diff --git a/tests/doc/comments.js b/tests/doc/comments.js
index 05bbfb9..8ef642e 100644
--- a/tests/doc/comments.js
+++ b/tests/doc/comments.js
@@ -261,6 +261,30 @@ describe('parse comments', () => {
   })
 
   describe('flow collection commens', () => {
+    test('line comment after , in seq', () => {
+      const doc = YAML.parseDocument(source`
+        [ a, #c0
+          b #c1
+        ]`)
+      expect(doc.contents.items).toMatchObject([
+        { value: 'a', comment: 'c0' },
+        { value: 'b', comment: 'c1' }
+      ])
+    })
+
+    test('line comment after , in map', () => {
+      const doc = YAML.parseDocument(source`
+        { a, #c0
+          b: c, #c1
+          d #c2
+        }`)
+      expect(doc.contents.items).toMatchObject([
+        { key: { value: 'a', comment: 'c0' } },
+        { key: { value: 'b' }, value: { value: 'c', comment: 'c1' } },
+        { key: { value: 'd', comment: 'c2' } }
+      ])
+    })
+
     test('multi-line comments', () => {
       const doc = YAML.parseDocument('{ a,\n#c0\n#c1\nb }')
       expect(doc.contents.items).toMatchObject([
@@ -446,6 +470,36 @@ describe('stringify comments', () => {
       `)
     })
   })
+
+  describe.skip('flow collection commens', () => {
+    test('line comment after , in seq', () => {
+      const doc = YAML.parseDocument(source`
+        [ a, #c0
+          b #c1
+        ]`)
+      expect(String(doc)).toBe(source`
+        [
+          a, #c0
+          b #c1
+        ]
+      `)
+    })
+
+    test('line comment after , in map', () => {
+      const doc = YAML.parseDocument(source`
+        { a, #c0
+          b: c, #c1
+          d #c2
+        }`)
+      expect(String(doc)).toBe(source`
+        {
+          ? a, #c0
+          b: c, #c1
+          ? d #c2
+        }
+      `)
+    })
+  })
 })
 
 describe('blank lines', () => {
diff --git a/tests/doc/errors.js b/tests/doc/errors.js
index 16f01ac..2d397af 100644
--- a/tests/doc/errors.js
+++ b/tests/doc/errors.js
@@ -122,24 +122,17 @@ describe('block collections', () => {
 describe('flow collections', () => {
   test('start only of flow map (eemeli/yaml#8)', () => {
     const doc = YAML.parseDocument('{')
-    const message = expect.stringContaining('Expected flow map to end with }')
-    expect(doc.errors).toMatchObject([{ message, offset: 1 }])
+    expect(doc.errors).toMatchObject([{ code: 'MISSING_CHAR', offset: 1 }])
   })
 
   test('start only of flow sequence (eemeli/yaml#8)', () => {
     const doc = YAML.parseDocument('[')
-    const message = expect.stringContaining(
-      'Expected flow sequence to end with ]'
-    )
-    expect(doc.errors).toMatchObject([{ message, offset: 1 }])
+    expect(doc.errors).toMatchObject([{ code: 'MISSING_CHAR', offset: 1 }])
   })
 
   test('flow sequence without end', () => {
     const doc = YAML.parseDocument('[ foo, bar,')
-    const message = expect.stringContaining(
-      'Expected flow sequence to end with ]'
-    )
-    expect(doc.errors).toMatchObject([{ message, offset: 11 }])
+    expect(doc.errors).toMatchObject([{ code: 'MISSING_CHAR', offset: 11 }])
   })
 
   test('doc-end within flow sequence', () => {
@@ -147,7 +140,7 @@ describe('flow collections', () => {
       prettyErrors: false
     })
     expect(doc.errors).toMatchObject([
-      { message: 'Expected flow sequence to end with ]' },
+      { code: 'MISSING_CHAR' },
       { message: 'Unexpected flow-seq-end token in YAML document: "]"' },
       {
         message:
@@ -166,18 +159,17 @@ describe('flow collections', () => {
 
   test('block seq in flow collection', () => {
     const doc = YAML.parseDocument('{\n- foo\n}')
-    expect(doc.errors).toHaveLength(1)
-    expect(doc.errors[0].message).toMatch(
-      'Block collections are not allowed within flow collections'
-    )
+    expect(doc.errors).toMatchObject([{ code: 'BLOCK_IN_FLOW' }])
   })
 
-  test('anchor before explicit key indicator', () => {
+  test('anchor before explicit key indicator in block map', () => {
+    const doc = YAML.parseDocument('&a ? A')
+    expect(doc.errors).toMatchObject([{ code: 'BAD_PROP_ORDER' }])
+  })
+
+  test('anchor before explicit key indicator in flow map', () => {
     const doc = YAML.parseDocument('{ &a ? A }')
-    expect(doc.errors).toHaveLength(1)
-    expect(doc.errors[0].message).toMatch(
-      'Anchors and tags must be after the ? indicator'
-    )
+    expect(doc.errors).toMatchObject([{ code: 'BAD_PROP_ORDER' }])
   })
 })
 
@@ -228,12 +220,14 @@ describe('pretty errors', () => {
     expect(docs[0].errors[0]).not.toHaveProperty('source')
     expect(docs[1].errors).toMatchObject([
       {
+        code: 'UNEXPECTED_TOKEN',
         message:
           'Unexpected , in flow map at line 3, column 7:\n\n{ 123,,, }\n      ^\n',
         offset: 16,
         linePos: { line: 3, col: 7 }
       },
       {
+        code: 'UNEXPECTED_TOKEN',
         message:
           'Unexpected , in flow map at line 3, column 8:\n\n{ 123,,, }\n       ^\n',
         offset: 17,
diff --git a/tests/doc/parse.js b/tests/doc/parse.js
index 2e7e468..c07c060 100644
--- a/tests/doc/parse.js
+++ b/tests/doc/parse.js
@@ -446,9 +446,9 @@ test('comment between key & : in flow collection (eemeli/yaml#149)', () => {
   expect(YAML.parse(src1)).toEqual({ a: 1 })
 
   const src2 = '{a\n#c\n:1}'
-  expect(() => YAML.parse(src2)).toThrow(
-    'Missing , between flow collection items'
-  )
+  expect(async () => YAML.parse(src2)).rejects.toMatchObject({
+    code: 'MISSING_CHAR'
+  })
 })
 
 describe('empty(ish) nodes', () => {