Repository: elastic/ecs

Issue:
All examples that are a composite type (array, object) should be quoted, to avoid YAML interpreting them
This issue mostly affects objects (like the example for `labels`). But it could also affect arrays, if for example we have an array that contains an unquoted `NO`, which would be interpreted by YAML as False, rather than the string "NO".

This issue not only affects the generated artifacts in the ECS repo, but also artifacts down the line, e.g. Beats starting from `generated/beats/fields.ecs.yml` and in turn generating various other artifacts.

- [x] #782 Immediate need is to fix `labels` to double quote it, an each values inside, to fix an issue in Beats
- [ ] #966 Fix all of the arrays
- [ ] #966 Add a linting rule or fail the build if any `example:` contains directly an array or an object, rather than a scalar.


Discussion:
++ this would also help the .NET client where I have had to handle examples by "santizing" them so they can be safely written into XML comments:

https://github.com/elastic/ecs-dotnet/blob/dc0d44fe1b0b3028a6a1cec044c7d8b30d4423a7/src/Generator/SchemaExtensionMethods.cs#L18-L23
This definitely has created issues with trying to parse ECS YAML (both the source schema, as well as the generated).

Basically, the field has to get specified as:

```
type Definition struct {
  // ... other fields
  Example          interface{}     `config:"example"`
}
```
So normalizing this certainly removes that ugly workaround. Thanks!



Resolving patch:
diff --git a/scripts/schema/cleaner.py b/scripts/schema/cleaner.py
--- a/scripts/schema/cleaner.py
+++ b/scripts/schema/cleaner.py
@@ -169,6 +169,7 @@ def field_assertions_and_warnings(field):
     if not ecs_helpers.is_intermediate(field):
         # check short description length if in strict mode
         single_line_short_description(field, strict=strict_mode)
+        check_example_value(field, strict=strict_mode)
         if field['field_details']['level'] not in ACCEPTABLE_FIELD_LEVELS:
             msg = "Invalid level for field '{}'.\nValue: {}\nAcceptable values: {}".format(
                 field['field_details']['name'], field['field_details']['level'],
@@ -193,3 +194,18 @@ def single_line_short_description(schema_or_field, strict=True):
             raise ValueError(msg)
         else:
             ecs_helpers.strict_warning(msg)
+
+
+def check_example_value(field, strict=True):
+    """
+    Checks if value of the example field is of type list or dict.
+    Fails or warns (depending on strict mode) if so.
+    """
+    example_value = field['field_details'].get('example', None)
+    if isinstance(example_value, (list, dict)):
+        name = field['field_details']['name']
+        msg = f"Example value for field `{name}` contains an object or array which must be quoted to avoid YAML interpretation."
+        if strict:
+            raise ValueError(msg)
+        else:
+            ecs_helpers.strict_warning(msg)


Test patch:
diff --git a/scripts/tests/unit/test_schema_cleaner.py b/scripts/tests/unit/test_schema_cleaner.py
--- a/scripts/tests/unit/test_schema_cleaner.py
+++ b/scripts/tests/unit/test_schema_cleaner.py
@@ -282,6 +282,64 @@ def test_multiline_short_description_warns_strict_disabled(self):
         except Exception:
             self.fail("cleaner.single_line_short_description() raised Exception unexpectedly.")
 
+    def test_field_example_value_is_object_raises(self):
+        field = {
+            'field_details': {
+                'name': 'test',
+                'example': {
+                    'a': 'bob',
+                    'b': 'alice'
+                }
+            }
+        }
+        with self.assertRaisesRegex(ValueError, 'contains an object or array'):
+            cleaner.check_example_value(field)
+
+    def test_field_example_value_is_array_raises(self):
+        field = {
+            'field_details': {
+                'name': 'test',
+                'example': [
+                    'bob',
+                    'alice'
+                ]
+            }
+        }
+        with self.assertRaisesRegex(ValueError, 'contains an object or array'):
+            cleaner.check_example_value(field)
+
+    def test_example_field_value_is_object_warns_strict_disabled(self):
+        field = {
+            'field_details': {
+                'name': 'test',
+                'example': {
+                    'a': 'bob',
+                    'b': 'alice'
+                }
+            }
+        }
+        try:
+            with self.assertWarnsRegex(UserWarning, 'contains an object or array'):
+                cleaner.check_example_value(field, strict=False)
+        except Exception:
+            self.fail("cleaner.check_example_value() raised Exception unexpectedly.")
+
+    def test_example_field_value_is_array_warns_strict_disabled(self):
+        field = {
+            'field_details': {
+                'name': 'test',
+                'example': [
+                    'bob',
+                    'alice'
+                ]
+            }
+        }
+        try:
+            with self.assertWarnsRegex(UserWarning, 'contains an object or array'):
+                cleaner.check_example_value(field, strict=False)
+        except Exception:
+            self.fail("cleaner.check_example_value() raised Exception unexpectedly.")
+
     def test_clean(self):
         '''A high level sanity test'''
         fields = self.schema_process()