Repository: valohai/valohai-yaml

Issue:
Indentation errors are cryptically linted
`vh lint` throws an error if the valohai.yaml file has lines that are not properly indented.

The error message is cryptic:
`error:   Type validation on pipeline: None is not of type 'object' (2.pipeline)`

The error message should clearly say that the issue is in indentation.

Below an example of a pipeline that fails. But if you indent everything under pipeline (lineno 2->) it's valid.

```yaml
- pipeline:
  name: mypipeline
  nodes:
    - name: validation-node
      type: task
      step: validate
    - name: ingest-node
      type: task
      step: ingest
      override:
      inputs:
          - name: image
  edges:
    - [validation-node.parameter.image, ingest-node.parameter.image]
```

Discussion:
codecov-commenter: ## [Codecov](https://app.codecov.io/gh/valohai/valohai-yaml/pull/94?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) Report
> Merging [#94](https://app.codecov.io/gh/valohai/valohai-yaml/pull/94?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) (5c15a91) into [master](https://app.codecov.io/gh/valohai/valohai-yaml/commit/00f85faa85b3ee5c45a04d6447f7b6902ecc2388?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) (00f85fa) will **decrease** coverage by `0.49%`.
> The diff coverage is `30.76%`.

```diff
@@            Coverage Diff             @@
##           master      #94      +/-   ##
==========================================
- Coverage   92.30%   91.82%   -0.49%     
==========================================
  Files          57       57              
  Lines        1651     1663      +12     
  Branches      280      282       +2     
==========================================
+ Hits         1524     1527       +3     
- Misses         67       75       +8     
- Partials       60       61       +1     
```


| [Impacted Files](https://app.codecov.io/gh/valohai/valohai-yaml/pull/94?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) | Coverage Δ | |
|---|---|---|
| [valohai\_yaml/lint.py](https://app.codecov.io/gh/valohai/valohai-yaml/pull/94?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV95YW1sL2xpbnQucHk=) | `81.42% <30.76%> (-11.68%)` | :arrow_down: |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai)


Patch:
diff --git a/valohai_yaml/lint.py b/valohai_yaml/lint.py
index 52ef31c..0df3be0 100644
--- a/valohai_yaml/lint.py
+++ b/valohai_yaml/lint.py
@@ -1,5 +1,6 @@
 from typing import Iterator, List, Optional
 
+import yaml as pyyaml
 from jsonschema.exceptions import relevance
 
 from valohai_yaml.excs import ValidationError
@@ -21,6 +22,9 @@ class LintResult:
     def add_warning(self, message: str, location: None = None, exception: Optional[Exception] = None) -> None:
         self.messages.append({'type': 'warning', 'message': message, 'location': location, 'exception': exception})
 
+    def add_hint(self, message: str, location: None = None, exception: Optional[Exception] = None) -> None:
+        self.messages.append({'type': 'hint', 'message': message, 'location': location, 'exception': exception})
+
     @property
     def warning_count(self) -> int:
         return sum(1 for m in self.messages if m['type'] == 'warning')
@@ -37,6 +41,10 @@ class LintResult:
     def errors(self) -> Iterator[LintResultMessage]:
         return (m for m in self.messages if m['type'] == 'error')
 
+    @property
+    def hints(self) -> Iterator[LintResultMessage]:
+        return (m for m in self.messages if m['type'] == 'hint')
+
     def is_valid(self) -> bool:
         return (self.warning_count == 0 and self.error_count == 0)
 
@@ -60,7 +68,17 @@ def lint_file(file_path: str) -> LintResult:
 def lint(yaml: YamlReadable) -> LintResult:
     lr = LintResult()
 
-    data = read_yaml(yaml)
+    try:
+        data = read_yaml(yaml)
+    except pyyaml.YAMLError as err:
+        if hasattr(err, 'problem_mark'):
+            mark = err.problem_mark
+            indent_error = f"Indentation Error at line {mark.line + 1}, column {mark.column + 1}"
+            lr.add_error(indent_error)
+        else:
+            lr.add_error(str(err))
+        return lr
+
     validator = get_validator()
     errors = sorted(
         validator.iter_errors(data),
@@ -79,6 +97,14 @@ def lint(yaml: YamlReadable) -> LintResult:
         styled_message = style(error.message, fg='red')
         styled_path = style('.'.join(obj_path), bold=True)
         lr.add_error(f'  {styled_validator} validation on {styled_schema_path}: {styled_message} ({styled_path})')
+        # when path has only 2 nodes. it means it has problem in main steps/pipelines/endpoints objects
+        if len(error.path) == 2 and not error.instance:
+            styled_hint = style(
+                "File contains valid YAML but there might be an indentation "
+                f"error in following configuration: {styled_path}",
+                fg='blue',
+            )
+            lr.add_hint(styled_hint)
 
     if len(errors) > 0:
         return lr


Test patch:
diff --git a/tests/error_examples/invalid-YAML-indentation.yaml b/tests/error_examples/invalid-YAML-indentation.yaml
new file mode 100644
index 0000000..b49c958
--- /dev/null
+++ b/tests/error_examples/invalid-YAML-indentation.yaml
@@ -0,0 +1,7 @@
+- step:
+  name: outdented
+    image: python:3.7
+    inputs: []
+    command:
+            - pip install -r requirements.txt
+            - python ./test.py {parameters}
diff --git a/tests/error_examples/invalid-indentation-with-valid-YAML.yaml b/tests/error_examples/invalid-indentation-with-valid-YAML.yaml
new file mode 100644
index 0000000..97489fe
--- /dev/null
+++ b/tests/error_examples/invalid-indentation-with-valid-YAML.yaml
@@ -0,0 +1,7 @@
+- step:
+  name: valid-yaml-but-unindented
+  image: python:3.7
+  inputs: []
+  command:
+          - pip install -r requirements.txt
+          - python ./test.py {parameters}
diff --git a/tests/test_linter.py b/tests/test_linter.py
index b6a6a67..19950af 100644
--- a/tests/test_linter.py
+++ b/tests/test_linter.py
@@ -2,7 +2,7 @@ from itertools import chain
 
 import pytest
 
-from tests.utils import get_warning_example_path
+from tests.utils import get_error_example_path, get_warning_example_path
 from valohai_yaml.lint import lint_file
 
 
@@ -22,3 +22,16 @@ def test_invalid_parameter_default(file, expected_message):
     items = lint_file(get_warning_example_path(file))
     messages = [item['message'] for item in chain(items.warnings, items.errors)]
     assert any(expected_message in message for message in messages), messages  # pragma: no branch
+
+
+@pytest.mark.parametrize('file, expected_message', [
+    ('invalid-indentation-with-valid-YAML.yaml',
+     '\x1b[34mFile contains valid YAML but there '
+     'might be an indentation error in following '
+     'configuration: \x1b[1m0.step\x1b'),
+    ('invalid-YAML-indentation.yaml', 'Indentation Error at line 3, column 10'),
+])
+def test_invalid_indentation(file, expected_message):
+    items = lint_file(get_error_example_path(file))
+    messages = [item['message'] for item in chain(items.hints, items.errors)]
+    assert any(expected_message in message for message in messages), messages  # pragma: no branch