Repository: adrienverge/yamllint

Issue:
Backslash-escaped non-printable character in double-quoted string throws exception when `quoted-strings` are `only-when-needed`
The following line of YAML:

```yaml
  echo_format: "\u001b[1;34m+ {command}\u001b[0m"
```

with the following .yamllint config:

```yaml
  quoted-strings:
    quote-type: any
    required: only-when-needed
```

Throws an exception (shown below).

Yamllint version:  yamllint 1.35.1
Python version:  Python 3.12.7

Dependency versions:

```
└── yamllint v1.35.1 (group: dev)
    ├── pathspec v0.12.1
    └── pyyaml v6.0.2
```

Reproducible problem documented at https://github.com/jmknoble/yamllint-quoted-escaped-nonprintable-char-problem .

The problem seems to be in   `yamllint.rules.quoted_strings._quotes_are_needed()` .  My hunch is that the already-decoded string is being passed, which `yaml.reader.check_printable()` interprets as unprintable.

Traceback:

```
Traceback (most recent call last):
  File "/Users/jmknoble/.local/bin/yamllint", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/cli.py", line 227, in run
    prob_level = show_problems(problems, file, args_format=args.format,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/cli.py", line 102, in show_problems
    for problem in problems:
                   ^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/linter.py", line 199, in _run
    for problem in get_cosmetic_problems(buffer, conf, filepath):
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/linter.py", line 136, in get_cosmetic_problems
    for problem in rule.check(rule_conf,
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/rules/quoted_strings.py", line 301, in check
    not _quotes_are_needed(token.value,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yamllint/rules/quoted_strings.py", line 212, in _quotes_are_needed
    loader = yaml.BaseLoader('key: ' + string)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yaml/loader.py", line 14, in __init__
    Reader.__init__(self, stream)
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yaml/reader.py", line 74, in __init__
    self.check_printable(stream)
  File "/Users/jmknoble/.local/share/uv/tools/yamllint/lib/python3.12/site-packages/yaml/reader.py", line 143, in check_printable
    raise ReaderError(self.name, position, ord(character),
yaml.reader.ReaderError: unacceptable character #x001b: special characters are not allowed
  in "<unicode string>", position 5
```

Discussion:
@adrienverge: Hello @jmknoble,

Thanks for the report!

I confirm the bug, reproducable with:
```bash
$ yamllint -d 'rules: {quoted-strings: {required: only-when-needed}}' - <<<'key: "\u001b"'
…
yaml.reader.ReaderError: unacceptable character #x001b: special characters are not allowed
  in "<unicode string>", position 5
```

A contribution to fix this would be welcome :+1:@jmknoble: > A contribution to fix this would be welcome

@adrienverge, let me know whether anything else is needed for the contributed fix in #703.

Solution patch:
diff --git a/yamllint/rules/quoted_strings.py b/yamllint/rules/quoted_strings.py
index b38f6ed..7cb88ff 100644
--- a/yamllint/rules/quoted_strings.py
+++ b/yamllint/rules/quoted_strings.py
@@ -204,11 +204,19 @@ def _quote_match(quote_type, token_style):
             (quote_type == 'double' and token_style == '"'))
 
 
-def _quotes_are_needed(string, is_inside_a_flow):
+def _quotes_are_needed(string, style, is_inside_a_flow):
     # Quotes needed on strings containing flow tokens
     if is_inside_a_flow and set(string) & {',', '[', ']', '{', '}'}:
         return True
 
+    if style == '"':
+        try:
+            yaml.reader.Reader('').check_printable('key: ' + string)
+        except yaml.reader.ReaderError:
+            # Special characters in a double-quoted string are assumed to have
+            # been backslash-escaped
+            return True
+
     loader = yaml.BaseLoader('key: ' + string)
     # Remove the 5 first tokens corresponding to 'key: ' (StreamStartToken,
     # BlockMappingStartToken, KeyToken, ScalarToken(value=key), ValueToken)
@@ -299,6 +307,7 @@ def check(conf, token, prev, next, nextnext, context):
         # Quotes are not strictly needed here
         if (token.style and tag == DEFAULT_SCALAR_TAG and token.value and
                 not _quotes_are_needed(token.value,
+                                       token.style,
                                        context['flow_nest_count'] > 0)):
             is_extra_required = any(re.search(r, token.value)
                                     for r in conf['extra-required'])


Test patch:
diff --git a/tests/rules/test_quoted_strings.py b/tests/rules/test_quoted_strings.py
index 1bcb6f8..31b392a 100644
--- a/tests/rules/test_quoted_strings.py
+++ b/tests/rules/test_quoted_strings.py
@@ -14,6 +14,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from tests.common import RuleTestCase
+import yaml.reader
 
 from yamllint import config
 
@@ -475,6 +476,30 @@ class QuotedValuesTestCase(RuleTestCase):
                    '- "foo bar"\n',
                    conf, problem1=(3, 3), problem2=(7, 3), problem3=(11, 3))
 
+    def test_only_when_needed_special_characters(self):
+        conf = 'quoted-strings: {required: only-when-needed}\n'
+        self.check('---\n'
+                   'k1: "\\u001b"\n',
+                   conf)
+        self.check('---\n'
+                   "k1: '\\u001b'\n",
+                   conf, problem=(2, 5))
+        self.check('---\n'
+                   'k1: \\u001b\n',
+                   conf)
+        self.assertRaises(yaml.reader.ReaderError, self.check,
+                          '---\n'
+                          'k1: "\u001b"\n',
+                          conf)
+        self.assertRaises(yaml.reader.ReaderError, self.check,
+                          '---\n'
+                          "k1: '\u001b'\n",
+                          conf)
+        self.assertRaises(yaml.reader.ReaderError, self.check,
+                          '---\n'
+                          'k1: \u001b\n',
+                          conf)
+
     def test_octal_values(self):
         conf = 'quoted-strings: {required: true}\n'