Repository: adrienverge/yamllint (language: python)

Issue:
yamllint allows referencing a non-existant anchor 
yamllint 1.26.1 (latest) does not flag references to anchors that don't exist.

```yaml
---
foo: &foo
    keyA: value1

bar:
    <<: *no_anchor_with_this_name
    keyB: value2
```

If you try to parse this using a tool like PyYAML it errors.

```
  File ".local/lib/python3.8/site-packages/ruamel/yaml/composer.py", line 115, in compose_node
    raise ComposerError(
ruamel.yaml.composer.ComposerError: found undefined alias 'no_anchor_with_this_name'
  in "test.yaml", line 6, column 9

```

Check for missing or duplicated anchors
Basic anchor\aliases checks, also fix for https://github.com/adrienverge/yamllint/issues/395.


Discussion / hints:
Hello,

Indeed yamllint does not currently detect inexistent anchors, or unmatching aliases.

It could be implemented in a new yamllint rule. In theory, it should be possible if aliases aren't used before anchors are declared (and this seems to be the case according to the [YAML specification 6.9.2.](https://yaml.org/spec/1.2/spec.html#id2785586): "*An anchor marks a node for *future* reference*").
I'd also like to check if there are unused anchors in the yaml files. Is this rule being considered for yamllint?

Yes, I think this is within the scope of a YAML linter. An option like `forbid-unused-anchors` could do the job.

In the end, a new rule like this would be great:

```yaml
rules:
  anchors:
    forbid-unknown-aliases: true      # true by default, because YAML spec fordids it
    forbid-duplicated-anchors: false  # false by default, because YAML spec allows it
    forbid-unused-anchors: false      # false by default, because YAML spec allows it
```
That's good to hear @adrienverge  . I like the example you gave. It gives more flexibility depending on the use case.

Not sure when we can expect those rules. I'm not familiar with the code base in this project and haven't implemented any custom rule. I can try open a PR if you can give some hints or guides on how to build those rules.

Answering https://github.com/adrienverge/yamllint/pull/420#discussion_r1096388404 from @amimas, about the `forbid-unused-anchors` option.

> Maybe we should discuss it in the feature request issue.

Sure :+1:

> I understand what you're saying and it makes sense. I was just asking because I wanted to look at the `forbid-unused-anchors` option. Without checking the entire file, not sure how to verify that an anchor isn't being used. Let me know if you have any suggestions.

I see. As discussed in the original comment, yamllint is designed to output errors right away, which allows:
\- having them sorted naturally,
\- being performant and avoid consuming too much RAM,
\- not losing past errors if the script crashes at some point.
Changing this would need a major refactor.

So, maybe a (not perfect) solution would be to output `forbid-unused-anchors` errors *at the end of the YAML stream* (`StreamEndToken` -- note that there can be multiple YAML documents inside the same file). The error report `anchor "foo" defined but unused` wouldn't match the line where `&foo` is defined, but would be positioned at the end of the YAML stream.
Wouldn't it be better to report it at the end of the *document* instead of stream?
> Wouldn't it be better to report it at the end of the _document_ instead of stream?

Sorry, that's what I meant but I twisted my words :man_facepalming: 
`DocumentEndToken` should be the correct token where to output these reports (not `StreamEndToken`).
> > Wouldn't it be better to report it at the end of the _document_ instead of stream?
> 
> Sorry, that's what I meant but I twisted my words 🤦‍♂️ `DocumentEndToken` should be the correct token where to output these reports (not `StreamEndToken`).

I think it's probably not that straight-forward. I tried implementing the `forbid-unused-anchors` (see #537). I'm new to Python. Appreciate your feedbacks in that PR.

While working on this, I also learned something new about yaml. While 3 dashes ( `---` ) represent start of a new document, apparently end of document is represented with 3 dots (`...`). See [section 2.2 of this doc](https://yaml.org/spec/1.2.2/). I haven't come across any yaml files yet that uses the "end of document" notation, but have seen multiple documents separated by "start of document" (i.e. 3 dashes). So, I think both `DocumentEndToken` and `StreamEndToken` should be used to report whether an anchor is unused or not.

That's true, `...` is part of the YAML standard (there is even a yamllint rule for it), but isn't commonly used.

> So, I think both DocumentEndToken and StreamEndToken should be used to report whether an anchor is unused or not.

Does this solution reliably indicates the end of a YAML document, even when there are multiple documents in a same file, and `...` is not present? Maybe there's a PyYAML token that is suited for that: could you check?

[![Coverage Status](https://coveralls.io/builds/56250524/badge)](https://coveralls.io/builds/56250524)

Coverage: 99.199% (+0.01%) from 99.184% when pulling **542e05fe8cdc7ebc453c07d8d82243b1db29f6f1 on newrushbolt:395-detect-missing-anchors** into **b9e1fd18c1acbaa4b0028196f8925ce4e91970d9 on adrienverge:master**.

Hello @adrienverge, thanks for the review. Sorry for the delay.
Read all the remarks, and realized that this PR had an infinite number of ways to be improved 🙈.

First of all, I decided to split this into two separate rules.
I did some rework, feel free to continue review.

1. Absolutely
2. Sure
3.  As well as `unknown-aliases` is correlates with YAML spec I think it should be enabled by default, and we should add a couple of words in the release note about it.
And I don't think we should revert this behavior even if the sh#t hits the fan after release. 
Talking about `anchor-duplicates`, it should probably stay disabled.

_Of course l'll squash all the commits after final approve._


@adrienverge is everything ok now?
If so can you merge this request?
@newrushbolt - are you able to look into the remaining feedbacks?
@newrushbolt we need feedback from you, because @amimas is implementing a new option for your new rule in https://github.com/adrienverge/yamllint/pull/537.

I know that yamllint requires high-quality contributions, so it's OK if you don't want to spend time on it. Someone else (maybe me) can implement this new rule in a new pull request. Please tell us what you prefer.

Fix patch:
diff --git a/docs/rules.rst b/docs/rules.rst
index c030c3db..eb3bc821 100644
--- a/docs/rules.rst
+++ b/docs/rules.rst
@@ -14,6 +14,11 @@ This page describes the rules and their options.
    :local:
    :depth: 1
 
+anchors
+-------
+
+.. automodule:: yamllint.rules.anchors
+
 braces
 ------
 
diff --git a/yamllint/conf/default.yaml b/yamllint/conf/default.yaml
index 0dea0aa4..b082e228 100644
--- a/yamllint/conf/default.yaml
+++ b/yamllint/conf/default.yaml
@@ -6,6 +6,7 @@ yaml-files:
   - '.yamllint'
 
 rules:
+  anchors: enable
   braces: enable
   brackets: enable
   colons: enable
diff --git a/yamllint/rules/__init__.py b/yamllint/rules/__init__.py
index 5a4fe811..6b5e4467 100644
--- a/yamllint/rules/__init__.py
+++ b/yamllint/rules/__init__.py
@@ -14,6 +14,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from yamllint.rules import (
+    anchors,
     braces,
     brackets,
     colons,
@@ -39,6 +40,7 @@
 )
 
 _RULES = {
+    anchors.ID: anchors,
     braces.ID: braces,
     brackets.ID: brackets,
     colons.ID: colons,
diff --git a/yamllint/rules/anchors.py b/yamllint/rules/anchors.py
new file mode 100644
index 00000000..6d0b1e60
--- /dev/null
+++ b/yamllint/rules/anchors.py
@@ -0,0 +1,118 @@
+# Copyright (C) 2023 Adrien Vergé
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+Use this rule to report duplicated anchors and aliases referencing undeclared
+anchors.
+
+.. rubric:: Options
+
+* Set ``forbid-undeclared-aliases`` to ``true`` to avoid aliases that reference
+  an anchor that hasn't been declared (either not declared at all, or declared
+  later in the document).
+* Set ``forbid-duplicated-anchors`` to ``true`` to avoid duplications of a same
+  anchor.
+
+.. rubric:: Default values (when enabled)
+
+.. code-block:: yaml
+
+ rules:
+   anchors:
+     forbid-undeclared-aliases: true
+     forbid-duplicated-anchors: false
+
+.. rubric:: Examples
+
+#. With ``anchors: {forbid-undeclared-aliases: true}``
+
+   the following code snippet would **PASS**:
+   ::
+
+    ---
+    - &anchor
+      foo: bar
+    - *anchor
+
+   the following code snippet would **FAIL**:
+   ::
+
+    ---
+    - &anchor
+      foo: bar
+    - *unknown
+
+   the following code snippet would **FAIL**:
+   ::
+
+    ---
+    - &anchor
+      foo: bar
+    - <<: *unknown
+      extra: value
+
+#. With ``anchors: {forbid-duplicated-anchors: true}``
+
+   the following code snippet would **PASS**:
+   ::
+
+    ---
+    - &anchor1 Foo Bar
+    - &anchor2 [item 1, item 2]
+
+   the following code snippet would **FAIL**:
+   ::
+
+    ---
+    - &anchor Foo Bar
+    - &anchor [item 1, item 2]
+"""
+
+
+import yaml
+
+from yamllint.linter import LintProblem
+
+
+ID = 'anchors'
+TYPE = 'token'
+CONF = {'forbid-undeclared-aliases': bool,
+        'forbid-duplicated-anchors': bool}
+DEFAULT = {'forbid-undeclared-aliases': True,
+           'forbid-duplicated-anchors': False}
+
+
+def check(conf, token, prev, next, nextnext, context):
+    if conf['forbid-undeclared-aliases'] or conf['forbid-duplicated-anchors']:
+        if isinstance(token, (yaml.StreamStartToken, yaml.DocumentStartToken)):
+            context['anchors'] = set()
+
+    if (conf['forbid-undeclared-aliases'] and
+            isinstance(token, yaml.AliasToken) and
+            token.value not in context['anchors']):
+        yield LintProblem(
+            token.start_mark.line + 1, token.start_mark.column + 1,
+            f'found undeclared alias "{token.value}"')
+
+    if (conf['forbid-duplicated-anchors'] and
+            isinstance(token, yaml.AnchorToken) and
+            token.value in context['anchors']):
+        yield LintProblem(
+            token.start_mark.line + 1, token.start_mark.column + 1,
+            f'found duplicated anchor "{token.value}"')
+
+    if conf['forbid-undeclared-aliases'] or conf['forbid-duplicated-anchors']:
+        if isinstance(token, yaml.AnchorToken):
+            context['anchors'].add(token.value)


Test patch:
diff --git a/tests/rules/test_anchors.py b/tests/rules/test_anchors.py
new file mode 100644
index 00000000..da1c5233
--- /dev/null
+++ b/tests/rules/test_anchors.py
@@ -0,0 +1,209 @@
+# Copyright (C) 2023 Adrien Vergé
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from tests.common import RuleTestCase
+
+
+class AnchorsTestCase(RuleTestCase):
+    rule_id = 'anchors'
+
+    def test_disabled(self):
+        conf = 'anchors: disable'
+        self.check('---\n'
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- &f_m {k: v}\n'
+                   '- &f_s [1, 2]\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '- *f_m\n'
+                   '- *f_s\n'
+                   '---\n'  # redeclare anchors in a new document
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &y 3, *x: 4, e: *y}\n'
+                   '...\n', conf)
+        self.check('---\n'
+                   '- &i 42\n'
+                   '---\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'    # declared in a previous document
+                   '- *f_m\n'  # never declared
+                   '- *f_m\n'
+                   '- *f_m\n'
+                   '- *f_s\n'  # declared after
+                   '- &f_s [1, 2]\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   '---\n'
+                   'block mapping 1: &b_m_bis\n'
+                   '  key: value\n'
+                   'block mapping 2: &b_m_bis\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &x 3, *x: 4, e: *y}\n'
+                   '...\n', conf)
+
+    def test_forbid_undeclared_aliases(self):
+        conf = ('anchors:\n'
+                '  forbid-undeclared-aliases: true\n'
+                '  forbid-duplicated-anchors: false\n')
+        self.check('---\n'
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- &f_m {k: v}\n'
+                   '- &f_s [1, 2]\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '- *f_m\n'
+                   '- *f_s\n'
+                   '---\n'  # redeclare anchors in a new document
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &y 3, *x: 4, e: *y}\n'
+                   '...\n', conf)
+        self.check('---\n'
+                   '- &i 42\n'
+                   '---\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'    # declared in a previous document
+                   '- *f_m\n'  # never declared
+                   '- *f_m\n'
+                   '- *f_m\n'
+                   '- *f_s\n'  # declared after
+                   '- &f_s [1, 2]\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   '---\n'
+                   'block mapping 1: &b_m_bis\n'
+                   '  key: value\n'
+                   'block mapping 2: &b_m_bis\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &x 3, *x: 4, e: *y}\n'
+                   '...\n', conf,
+                   problem1=(9, 3),
+                   problem2=(10, 3),
+                   problem3=(11, 3),
+                   problem4=(12, 3),
+                   problem5=(13, 3),
+                   problem6=(24, 7),
+                   problem7=(27, 36))
+
+    def test_forbid_duplicated_anchors(self):
+        conf = ('anchors:\n'
+                '  forbid-undeclared-aliases: false\n'
+                '  forbid-duplicated-anchors: true\n')
+        self.check('---\n'
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- &f_m {k: v}\n'
+                   '- &f_s [1, 2]\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '- *f_m\n'
+                   '- *f_s\n'
+                   '---\n'  # redeclare anchors in a new document
+                   '- &b true\n'
+                   '- &i 42\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'
+                   '- *s\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &y 3, *x: 4, e: *y}\n'
+                   '...\n', conf)
+        self.check('---\n'
+                   '- &i 42\n'
+                   '---\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &b true\n'
+                   '- &s hello\n'
+                   '- *b\n'
+                   '- *i\n'    # declared in a previous document
+                   '- *f_m\n'  # never declared
+                   '- *f_m\n'
+                   '- *f_m\n'
+                   '- *f_s\n'  # declared after
+                   '- &f_s [1, 2]\n'
+                   '---\n'
+                   'block mapping: &b_m\n'
+                   '  key: value\n'
+                   '---\n'
+                   'block mapping 1: &b_m_bis\n'
+                   '  key: value\n'
+                   'block mapping 2: &b_m_bis\n'
+                   '  key: value\n'
+                   'extended:\n'
+                   '  <<: *b_m\n'
+                   '  foo: bar\n'
+                   '---\n'
+                   '{a: 1, &x b: 2, c: &x 3, *x: 4, e: *y}\n'
+                   '...\n', conf,
+                   problem1=(5, 3),
+                   problem2=(6, 3),
+                   problem3=(21, 18),
+                   problem4=(27, 20))