Repository: carbon-language/carbon-lang
File: toolchain/lex/string_literal.h (C/C++)

Code before:
  enum class Kind : int8_t {
    // A character literal is still handled through string literal lexing.
    // Note the consumer must still validate as a single character.
    Char,

    // A single-line string, `"<content>"`.
    SingleLine,

    // A multi-line string, `'''<content>'''`.
    MultiLine,

    // An incorrectly double-quoted multi-line string, `"""<content>"""`.
    MultiLineWithDoubleQuotes,
  };

  // Extract a string literal token from the given text, if it has a suitable
  // form. Returning std::nullopt indicates no string literal was found;
  // returning an invalid literal indicates a string prefix was found, but it's
  // malformed and is returning a partial string literal to assist error
  // construction.
  static auto Lex(llvm::StringRef source_text) -> std::optional<StringLiteral>;

  // Expand any escape sequences and compute the resulting character. This
  // handles error recovery internally, but can return nullopt for an invalid
  // character.
  auto ComputeCharValue(Diagnostics::Emitter<const char*>& emitter) const
      -> std::optional<CharLiteralValue>;

  // Expand any escape sequences in the given string literal and compute the
  // resulting value. This handles error recovery internally and cannot fail.
  //
  // When content_needs_validation_ is false and the string has no indent to
  // deal with, this can return the content directly. Otherwise, the allocator
  // will be used for the StringRef.
  auto ComputeStringValue(llvm::BumpPtrAllocator& allocator,
                          Diagnostics::Emitter<const char*>& emitter) const
      -> llvm::StringRef;

  // Get the text corresponding to this literal.
  auto text() const -> llvm::StringRef { return text_; }

  // Determine whether this is a multi-line string literal.
  auto kind() const -> Kind { return kind_; }

  // Returns true if the string has a valid terminator.
  auto is_terminated() const -> bool { return is_terminated_; }

 private:
  struct Introducer;

  explicit StringLiteral(llvm::StringRef text, llvm::StringRef content,

Review comment (refactor):
Hmm, I wonder if we want to use the [Unicode replacement character](https://en.wikipedia.org/wiki/Specials_(Unicode_block)) U+FFFD to avoid even needing nullopt?

Maybe a TODO, doesn't seem blocking unless you'd rather that to simplify code.

Code after:
  enum class Kind : int8_t {
    // A character literal is still handled through string literal lexing.
    Char,

    // A single-line string, `"<content>"`.
    SingleLine,

    // A multi-line string, `'''<content>'''`.
    MultiLine,

    // An incorrectly double-quoted multi-line string, `"""<content>"""`.
    MultiLineWithDoubleQuotes,
  };

  // Extract a string literal token from the given text, if it has a suitable
  // form. Returning std::nullopt indicates no string literal was found;
  // returning an invalid literal indicates a string prefix was found, but it's
  // malformed and is returning a partial string literal to assist error
  // construction.
  static auto Lex(llvm::StringRef source_text) -> std::optional<StringLiteral>;

  // Expand any escape sequences and compute the resulting character. This
  // handles error recovery internally, but can return nullopt for an invalid
  // character.
  auto ComputeCharValue(Diagnostics::Emitter<const char*>& emitter) const
      -> std::optional<CharLiteralValue>;

  // Expand any escape sequences in the given string literal and compute the
  // resulting value. This handles error recovery internally and cannot fail.
  //
  // When content_needs_validation_ is false and the string has no indent to
  // deal with, this can return the content directly. Otherwise, the allocator
  // will be used for the StringRef.
  auto ComputeStringValue(llvm::BumpPtrAllocator& allocator,
                          Diagnostics::Emitter<const char*>& emitter) const
      -> llvm::StringRef;

  // Get the text corresponding to this literal.
  auto text() const -> llvm::StringRef { return text_; }

  // Determine whether this is a multi-line string literal.
  auto kind() const -> Kind { return kind_; }

  // Returns true if the string has a valid terminator.
  auto is_terminated() const -> bool { return is_terminated_; }

 private:
  struct Introducer;

  explicit StringLiteral(llvm::StringRef text, llvm::StringRef content,
                         bool content_needs_validation, int hash_level,