Repository: ansible-community/antsibull-changelog

Issue:
Please add config option to disable generating TOC
When using certain mkdocs themes, the "Topics" (Table of Contents) can be redundant with the same information provided along the side bar.

# Topics generated
![Image](https://github.com/user-attachments/assets/20585d46-e7e0-4327-ab7c-520a686edb04)

# Topics removed
![Image](https://github.com/user-attachments/assets/fd5dca5f-1ea8-410b-b67c-167df9d23899)

I'd like to see an option to disable generating this section of the changelog file.

Discussion:
@felixfontein: I've been thinking about this a bit, including your PR (#191, thanks for that!). I think it definitely makes sense to add more configuration settings like this. Already in this example, there are two levels you can have TOCs in - global and per release; right now the global one is always on, the other always off, with no way to change this. (TOCs on release level are for example enabled in Ansible's changelog, see for example https://github.com/ansible-community/ansible-build-data/blob/main/10/CHANGELOG-v10.md#v10-7-0.)

But I'm not sure whether a global flag "enable/disable top-level TOC" is a good idea. Maybe you want the TOC to be gone for the MD version you want to include in your mkdocs page, but not the MD file you want to have in your repository's root on GitHub (like https://github.com/ansible-collections/community.general/blob/stable-10/CHANGELOG.md, where you can enable an outline, but it's not on by default, at least for me). Or you want the RST version to be different from MD in this regard (RST for a Sphinx page, MD for GitHub).

I think it would be best to have an alternative way to specify output files with (some) individual settings for them, including filename, format, global TOC, and per-release TOC. Something like (with more options for TOC depth):
```yaml
outputs:
  - file: changelogs/CHANGELOG-v%s.rst
    filename_version_depth: 2
    format: rst
    global_toc: true
    global_toc_depth: 1
    per_release_toc: true
    per_release_toc_depth: 1
  - file: CHANGELOG.md
    format: md
    global_toc: true
    global_toc_depth: 2
    per_release_toc: false
  - file: docsite/CHANGELOG.md
    format: md
    global_toc: false
    per_release_toc: false
```
This would also make the options `changelog_filename_template` and `changelog_filename_version_depth` (both mostly historic and with specific values for backwards compatibility) and `output_formats` obsolete.@joewesch: That makes sense. However, that seems to require a much deeper dive into the code base, and therefore a much longer time commitment, than I can afford to spend on this. If that is the path you want to go, feel free to close my PR and I guess I'll wait for you or someone else to complete the FR.@felixfontein: That's very understandable! I'll likely take a look at that on Thursday on my train ride back from CfgMgmtCamp...@felixfontein: Sorry, I didn't got around to properly work on that yet... On that Thursday I started getting sick and was out for almost a week, and then other things have been going on. I haven't forgotten about this and will take a look hopefully soon :)@felixfontein: I've started implementing this in #194.

Solution patch:
diff --git a/changelogs/fragments/193-config.yml b/changelogs/fragments/193-config.yml
new file mode 100644
index 0000000..ed3fb21
--- /dev/null
+++ b/changelogs/fragments/193-config.yml
@@ -0,0 +1,10 @@
+minor_changes:
+  - "Antsibull-changelog now uses Pydantic to parse and validate the config.
+     This means that validation is more strict than before and might reject configs that were incorrect, but still got accepted somehow
+     (https://github.com/ansible-community/antsibull-changelog/pull/193)."
+  - "Antsibull-changelog now depends on Pydantic 2 (https://github.com/ansible-community/antsibull-changelog/pull/193)."
+breaking_changes:
+  - "When using antsibull-changelog as a library, ``ChangelogConfig``'s constructor should no longer be called directly.
+     Instead, use the class method ``ChangelogConfig.parse()``, which has the same signature than the previous constructor,
+     except that ``ignore_is_other_project`` now must be a keyword parameter
+     (https://github.com/ansible-community/antsibull-changelog/pull/193)."
diff --git a/pyproject.toml b/pyproject.toml
index 5681b89..19f63ef 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -42,6 +42,7 @@ dependencies = [
     "rstcheck >= 3.0.0, < 7.0.0",
     "semantic_version",
     "antsibull-docutils >= 1.0.0, < 2.0.0",
+    "pydantic ~= 2.0",
 ]
 
 [project.urls]
diff --git a/src/antsibull_changelog/cli.py b/src/antsibull_changelog/cli.py
index c465346..b169457 100644
--- a/src/antsibull_changelog/cli.py
+++ b/src/antsibull_changelog/cli.py
@@ -341,7 +341,7 @@ def run(args: list[str]) -> int:
 
         return arguments.func(arguments)
     except ChangelogError as e:
-        LOGGER.error(str(e))
+        LOGGER.error("{}", str(e))
         if verbosity > 2:
             traceback.print_exc()
         return C.RC_COMMAND_FAILED
diff --git a/src/antsibull_changelog/config.py b/src/antsibull_changelog/config.py
index 1c36dfc..f6d1e1c 100644
--- a/src/antsibull_changelog/config.py
+++ b/src/antsibull_changelog/config.py
@@ -11,18 +11,36 @@ Configuration classes for paths and changelogs.
 
 from __future__ import annotations
 
-import collections
 import enum
 import os
+import typing as t
 from collections.abc import Mapping
-from typing import Literal
+from collections.abc import Sequence as _Sequence
 
+import pydantic as p
 from antsibull_fileutils.yaml import load_yaml_file, store_yaml_file
 
 from .errors import ChangelogError
 from .logger import LOGGER
 
 
+def _is_sequence(value: t.Any) -> bool:
+    if not isinstance(value, _Sequence):
+        return False
+    return not isinstance(value, (str, bytes))
+
+
+def _ordinal(index: int) -> str:
+    one = abs(index) % 10
+    if one == 1:
+        return f"{index}st"
+    if one == 2:
+        return f"{index}nd"
+    if one == 3:
+        return f"{index}rd"
+    return f"{index}th"
+
+
 class TextFormat(enum.Enum):
     """
     Supported text formats.
@@ -39,7 +57,7 @@ class TextFormat(enum.Enum):
             return "rst"
         if self == TextFormat.MARKDOWN:
             return "md"
-        raise ValueError(f"Unknown text format {self}")
+        raise ValueError(f"Unknown text format {self}")  # pragma: no cover
 
     @staticmethod
     def from_extension(extension: str) -> TextFormat:
@@ -267,13 +285,13 @@ class CollectionDetails:
             )
 
         if what_for is None:
-            what_for = 'load field "{0}"'.format(needed_var)
+            what_for = f'load field "{needed_var}"'
         try:
             galaxy_yaml = load_galaxy_metadata(self.paths)
         except Exception as exc:
-            msg = "Cannot find galaxy.yml to {0}: {1}".format(what_for, exc)
+            msg = f"Cannot find galaxy.yml to {what_for}: {exc}"
             if help_text is not None:
-                msg = "{0}. {1}".format(msg, help_text)
+                msg = f"{msg}. {help_text}"
             raise ChangelogError(msg) from exc
 
         self._parse_galaxy_yaml(galaxy_yaml)
@@ -344,175 +362,240 @@ DEFAULT_SECTIONS = [
 ]
 
 
-class ChangelogConfig:
-    # pylint: disable=too-many-instance-attributes
+class ChangelogConfig(p.BaseModel):
     """
     Configuration for changelogs.
     """
 
+    model_config = p.ConfigDict(
+        frozen=False, extra="allow", validate_default=True, arbitrary_types_allowed=True
+    )
+
     paths: PathsConfig
     collection_details: CollectionDetails
-
-    config: dict
     is_collection: bool
-    title: str | None
-    notes_dir: str
-    prelude_name: str
-    prelude_title: str
-    new_plugins_after_name: str
-    changes_file: str
-    changes_format: str | None
-    keep_fragments: bool
-    prevent_known_fragments: bool
-    use_fqcn: bool
-    archive_path_template: str | None
-    changelog_filename_template: str
-    changelog_filename_version_depth: int
-    mention_ancestor: bool
-    trivial_section_name: str | None
-    release_tag_re: str
-    pre_release_tag_re: str
-    always_refresh: str
-    ignore_other_fragment_extensions: bool
-    sanitize_changelog: bool
-    flatmap: bool | None
-    use_semantic_versioning: bool
-    is_other_project: bool
-    sections: Mapping[str, str]
-    output_formats: set[TextFormat]
-    add_plugin_period: bool
-    changelog_nice_yaml: bool
-    changelog_sort: Literal[
+    config: dict
+
+    title: t.Optional[str] = None
+    notes_dir: str = p.Field(default="fragments", alias="notesdir")
+    prelude_name: str = p.Field(default="release_summary", alias="prelude_section_name")
+    prelude_title: str = p.Field(
+        default="Release Summary", alias="prelude_section_title"
+    )
+    new_plugins_after_name: str = ""  # not used
+    changes_file: str = ".changes.yaml"
+    changes_format: t.Literal["combined"]
+    keep_fragments: bool = False
+    prevent_known_fragments: bool  # default is set in parse()
+    use_fqcn: bool = False
+    archive_path_template: t.Optional[str] = None
+    changelog_filename_template: str = "CHANGELOG-v%s.rst"
+    changelog_filename_version_depth: int = 2
+    mention_ancestor: bool = True
+    trivial_section_name: t.Optional[str]  # default is set by parse()
+    release_tag_re: str = r"((?:[\d.ab]|rc)+)"  # only relevant for ansible-core
+    pre_release_tag_re: str = (
+        r"(?P<pre_release>\.\d+(?:[ab]|rc)+\d*)$"  # only relevant for ansible-core
+    )
+    always_refresh: str = "none"
+    ignore_other_fragment_extensions: bool = False
+    sanitize_changelog: bool = False
+    flatmap: t.Optional[bool] = None
+    use_semantic_versioning: bool = (
+        True  # default is False for ansible-core and other projects
+    )
+    is_other_project: bool = False
+    sections: Mapping[str, str] = dict(DEFAULT_SECTIONS)
+    output_formats: set[TextFormat] = {TextFormat.RESTRUCTURED_TEXT}
+    add_plugin_period: bool = False
+    changelog_nice_yaml: bool = False
+    changelog_sort: t.Literal[
         "unsorted",
         "version",
         "version_reversed",
         "alphanumerical",
-    ]
-    vcs: Literal["none", "auto", "git"]
+    ] = "alphanumerical"
+    vcs: t.Literal["none", "auto", "git"] = "none"
+
+    @p.field_validator("always_refresh", mode="before")
+    @classmethod
+    def fix_always_refresh(cls, value: t.Any) -> str:
+        """
+        Validate and adjust the value of always_refresh.
+        """
+        if value is True:
+            value = "full"
+        elif value is False:
+            value = "none"
+        if not isinstance(value, str):
+            raise ValueError(
+                "If specified, always_refresh must be a boolean or a string"
+            )
+        if value not in {"full", "none"}:
+            for part in value.split(","):
+                part = part.strip()
+                if part not in {
+                    "plugins",
+                    "plugins-without-removal",
+                    "fragments",
+                    "fragments-without-archives",
+                }:
+                    if part in {"full", "none"}:
+                        raise ValueError(
+                            f'The config value always_refresh must not contain "{part}"'
+                            " together with other values"
+                        )
+                    raise ValueError(
+                        f'The config value always_refresh contains an invalid value "{part}"'
+                    )
+        return value
 
-    def __init__(
-        self,
-        paths: PathsConfig,
-        collection_details: CollectionDetails,
-        config: dict,
-        ignore_is_other_project: bool = False,
-    ):
+    @p.field_validator("sections", mode="before")
+    @classmethod
+    def parse_sections(cls, value: t.Any) -> Mapping[str, str]:
         """
-        Create changelog config from dictionary.
+        Parse the value of sections.
         """
-        self.paths = paths
-        self.collection_details = collection_details
-        self.config = config
-
-        self.is_collection = paths.is_collection
-        self.title = self.config.get("title")
-        self.notes_dir = self.config.get("notesdir", "fragments")
-        self.prelude_name = self.config.get("prelude_section_name", "release_summary")
-        self.prelude_title = self.config.get("prelude_section_title", "Release Summary")
-        self.new_plugins_after_name = self.config.get(
-            "new_plugins_after_name", ""
-        )  # not used
-        self.changes_file = self.config.get("changes_file", ".changes.yaml")
-        self.changes_format = self.config.get("changes_format")
-        self.keep_fragments = self.config.get("keep_fragments", False)
-        self.prevent_known_fragments = self.config.get(
-            "prevent_known_fragments", self.keep_fragments
-        )
-        self.use_fqcn = self.config.get("use_fqcn", False)
-        self.archive_path_template = self.config.get("archive_path_template")
-        self.changelog_filename_template = self.config.get(
-            "changelog_filename_template", "CHANGELOG-v%s.rst"
-        )
-        self.changelog_filename_version_depth = self.config.get(
-            "changelog_filename_version_depth", 2
-        )
-        self.mention_ancestor = self.config.get("mention_ancestor", True)
-        has_trivial_section_by_default = paths.is_collection or paths.is_other_project
-        self.trivial_section_name = self.config.get(
-            "trivial_section_name",
-            "trivial" if has_trivial_section_by_default else None,
-        )
-        self.sanitize_changelog = self.config.get("sanitize_changelog", False)
-        always_refresh = self.config.get("always_refresh", False)
-        if always_refresh is True:
-            always_refresh = "full"
-        if always_refresh is False:
-            always_refresh = "none"
-        self.always_refresh = always_refresh
-        self.ignore_other_fragment_extensions = self.config.get(
-            "ignore_other_fragment_extensions", False
-        )
-        self.flatmap = self.config.get("flatmap")
-        self.use_semantic_versioning = True
-        self.is_other_project = self.config.get("is_other_project", False)
-
-        # The following are only relevant for ansible-core/-base and other projects:
-        self.release_tag_re = self.config.get("release_tag_re", r"((?:[\d.ab]|rc)+)")
-        self.pre_release_tag_re = self.config.get(
-            "pre_release_tag_re", r"(?P<pre_release>\.\d+(?:[ab]|rc)+\d*)$"
-        )
-        if not self.is_collection:
-            self.use_semantic_versioning = self.config.get(
-                "use_semantic_versioning", False
-            )
-
-        sections = collections.OrderedDict([(self.prelude_name, self.prelude_title)])
-        for section_name, section_title in self.config.get(
-            "sections", DEFAULT_SECTIONS
-        ):
-            sections[section_name] = section_title
-        self.sections = sections
-
-        self.output_formats = set()
-        for extension in self.config.get("output_formats", ["rst"]):
+        if isinstance(value, Mapping):
+            for k, v in value.items():
+                if not isinstance(k, str) or not isinstance(v, str):
+                    raise ValueError(
+                        "The config value sections must be a dictionary mapping strings to strings"
+                    )
+            return value
+        if not _is_sequence(value):
+            raise ValueError("The config value sections must be a list")
+        sections = {}
+        for index, entry in enumerate(value):
+            if not _is_sequence(entry) or len(entry) != 2:
+                raise ValueError(
+                    f"The {_ordinal(index + 1)} entry of config value sections"
+                    " must be a list of length 2"
+                )
+            if not isinstance(entry[0], str):
+                raise ValueError(
+                    f"The {_ordinal(index + 1)} entry of config value sections"
+                    " does not have a string as the first element"
+                )
+            if not isinstance(entry[1], str):
+                raise ValueError(
+                    f"The {_ordinal(index + 1)} entry of config value sections"
+                    " does not have a string as the second element"
+                )
+            if entry[0] in sections:
+                raise ValueError(
+                    f"The section name {entry[0]!r} appears more than once"
+                )
+            sections[entry[0]] = entry[1]
+        return sections
+
+    @p.field_validator("output_formats", mode="before")
+    @classmethod
+    def parse_output_formats(cls, value: t.Any) -> set[TextFormat]:
+        """
+        Parse the value of output_formats.
+        """
+        if isinstance(value, set):
+            for entry in value:
+                if not isinstance(entry, TextFormat):
+                    raise ValueError("The config value output_formats must be a list")
+            return value
+        if not _is_sequence(value):
+            raise ValueError("The config value output_formats must be a list")
+        result: set[TextFormat] = set()
+        for index, entry in enumerate(value):
+            if not isinstance(entry, str):
+                raise ValueError(
+                    f"The {_ordinal(index + 1)} entry of config value output_formats"
+                    " is not a string"
+                )
             try:
-                self.output_formats.add(TextFormat.from_extension(extension))
+                text_format = TextFormat.from_extension(entry)
             except ValueError as exc:
-                raise ChangelogError(
-                    f"Found unknown extension in output_formats: {exc}"
+                raise ValueError(
+                    f"The {_ordinal(index + 1)} entry of config value output_formats"
+                    f" is an unknown extension: {exc}"
                 ) from exc
+            if text_format in result:
+                raise ValueError(f"The output format {entry!r} appears more than once")
+            result.add(text_format)
+        return result
 
-        self.add_plugin_period = self.config.get("add_plugin_period", False)
-
-        self.changelog_nice_yaml = self.config.get("changelog_nice_yaml", False)
-
-        self.changelog_sort = self.config.get("changelog_sort", "alphanumerical")
-
-        self.vcs = self.config.get("vcs", "none")
+    @p.model_validator(mode="after")
+    def postprocess_sections(self) -> t.Self:
+        """
+        Postprocess sections value by inserting the prelude section.
+        """
+        sections = {self.prelude_name: self.prelude_title}
+        for k, v in self.sections.items():
+            if k == self.prelude_name:
+                raise ValueError(
+                    f"No section name must equal prelude_name ({self.prelude_name!r})"
+                )
+            sections[k] = v
+        self.sections = sections
+        return self
 
-        self._validate_config(ignore_is_other_project)
+    @p.model_validator(mode="after")
+    def validate_use_semantic_versioning(self) -> t.Self:
+        """
+        Validate use_semantic_versioning for collections.
+        """
+        if self.is_collection and not self.use_semantic_versioning:
+            raise ValueError(
+                "The config value use_semantic_versioning must be true for collections"
+            )
+        return self
 
-    def _validate_config(self, ignore_is_other_project: bool) -> None:
+    def _validate_other_project(self) -> None:
         """
         Basic config validation.
         """
-        if (
-            self.is_other_project != self.paths.is_other_project
-            and not ignore_is_other_project
-        ):
-            raise ChangelogError(
-                "is_other_project must be {0}".format(self.is_other_project)
+        if self.is_other_project and self.is_collection:
+            raise ChangelogError("is_other_project must not be true for collections")
+        if self.is_other_project != self.paths.is_other_project:
+            raise ChangelogError(f"is_other_project must be {self.is_other_project}")
+
+    @classmethod
+    def parse(
+        cls,
+        paths: PathsConfig,
+        collection_details: CollectionDetails,
+        config: dict,
+        *,
+        ignore_is_other_project: bool = False,
+    ) -> ChangelogConfig:
+        """
+        Parse a raw changelog configuration.
+        """
+        adjusted_config = config.copy()
+        adjusted_config["paths"] = paths
+        adjusted_config["collection_details"] = collection_details
+        adjusted_config["is_collection"] = paths.is_collection
+        adjusted_config["config"] = config
+
+        # Set some special defaults
+        if "trivial_section_name" not in adjusted_config:
+            adjusted_config["trivial_section_name"] = (
+                "trivial" if (paths.is_collection or paths.is_other_project) else None
             )
-        if self.is_other_project and self.is_collection and not ignore_is_other_project:
-            raise ChangelogError("is_other_project must not be True for collections")
-        if self.changes_format != "combined":
-            raise ChangelogError('changes_format must be "combined"')
-
-        valid_sort_options = [
-            "unsorted",
-            "version",
-            "version_reversed",
-            "alphanumerical",
-        ]
-        if self.changelog_sort not in valid_sort_options:
-            raise ChangelogError(
-                f"Invalid changelog_sort option: {self.changelog_sort}"
+        if "use_semantic_versioning" not in adjusted_config and not paths.is_collection:
+            adjusted_config["use_semantic_versioning"] = False
+        if "prevent_known_fragments" not in adjusted_config:
+            adjusted_config["prevent_known_fragments"] = adjusted_config.get(
+                "keep_fragments", False
             )
 
-        if self.vcs not in ("none", "git", "auto"):
-            raise ChangelogError(
-                f"Invalid VCS value {self.vcs!r}. Must be one of none, git, and auto."
-            )
+        # Parse
+        try:
+            result = cls.model_validate(adjusted_config)
+        except Exception as exc:
+            raise ChangelogError(f"Error while parsing changlog config: {exc}") from exc
+
+        if not ignore_is_other_project:
+            result._validate_other_project()  # pylint: disable=protected-access
+
+        return result
 
     def store(self) -> None:  # noqa: C901
         """
@@ -585,8 +668,8 @@ class ChangelogConfig:
         """
         config = load_yaml_file(paths.config_path)
         if not isinstance(config, dict):
-            raise ChangelogError("{0} must be a dictionary".format(paths.config_path))
-        return ChangelogConfig(
+            raise ChangelogError(f"{paths.config_path} must be a dictionary")
+        return ChangelogConfig.parse(
             paths,
             collection_details,
             config,
@@ -624,4 +707,4 @@ class ChangelogConfig:
         if paths.is_other_project:
             config["is_other_project"] = True
             config["use_semantic_versioning"] = True
-        return ChangelogConfig(paths, collection_details, config)
+        return ChangelogConfig.parse(paths, collection_details, config)


Test patch:
diff --git a/tests/units/test_config.py b/tests/units/test_config.py
index 32a3b8b..20d5e0d 100644
--- a/tests/units/test_config.py
+++ b/tests/units/test_config.py
@@ -10,6 +10,7 @@ Test config module.
 from __future__ import annotations
 
 import os
+import re
 
 import pytest
 
@@ -20,52 +21,62 @@ from antsibull_changelog.errors import ChangelogError
 @pytest.fixture
 def root():
     old_cwd = os.getcwd()
-    os.chdir("/")
-    yield
-    os.chdir(old_cwd)
+    try:
+        os.chdir("/")
+        yield
+    finally:
+        os.chdir(old_cwd)
 
 
 @pytest.fixture
 def cwd_tmp_path(tmp_path):
     old_cwd = os.getcwd()
-    os.chdir(tmp_path)
-    yield tmp_path
-    os.chdir(old_cwd)
+    try:
+        os.chdir(tmp_path)
+        yield tmp_path
+    finally:
+        os.chdir(old_cwd)
 
 
 @pytest.fixture
 def ansible_config_path(tmp_path):
     old_cwd = os.getcwd()
-    os.chdir(tmp_path)
-    d = tmp_path / "lib"
-    d.mkdir()
-    d = d / "ansible"
-    d.mkdir()
-    d = tmp_path / "changelogs"
-    d.mkdir()
-    yield d / "config.yaml"
-    os.chdir(old_cwd)
+    try:
+        os.chdir(tmp_path)
+        d = tmp_path / "lib"
+        d.mkdir()
+        d = d / "ansible"
+        d.mkdir()
+        d = tmp_path / "changelogs"
+        d.mkdir()
+        yield d / "config.yaml"
+    finally:
+        os.chdir(old_cwd)
 
 
 @pytest.fixture
 def collection_config_path(tmp_path):
     old_cwd = os.getcwd()
-    os.chdir(tmp_path)
-    (tmp_path / "galaxy.yml").write_text("")
-    d = tmp_path / "changelogs"
-    d.mkdir()
-    yield d / "config.yaml"
-    os.chdir(old_cwd)
+    try:
+        os.chdir(tmp_path)
+        (tmp_path / "galaxy.yml").write_text("")
+        d = tmp_path / "changelogs"
+        d.mkdir()
+        yield d / "config.yaml"
+    finally:
+        os.chdir(old_cwd)
 
 
 @pytest.fixture
 def other_config_path(tmp_path):
     old_cwd = os.getcwd()
-    os.chdir(tmp_path)
-    d = tmp_path / "changelogs"
-    d.mkdir()
-    yield d / "config.yaml"
-    os.chdir(old_cwd)
+    try:
+        os.chdir(tmp_path)
+        d = tmp_path / "changelogs"
+        d.mkdir()
+        yield d / "config.yaml"
+    finally:
+        os.chdir(old_cwd)
 
 
 def test_detect_complete_fail(root):
@@ -287,6 +298,14 @@ def test_collection_details(tmp_path):
         details.get_flatmap()
     assert "Cannot find galaxy.yml" in str(exc.value)
 
+    galaxy_path = tmp_path / "galaxy.yml"
+    galaxy_path.write_text("---\nfoo")
+    paths = PathsConfig.force_collection(str(tmp_path))
+    details = CollectionDetails(paths)
+    with pytest.raises(ChangelogError) as exc:
+        details.get_namespace()
+    assert "galaxy.yml must be a dictionary" in str(exc.value)
+
     galaxy_path = tmp_path / "galaxy.yml"
     galaxy_path.write_text("---\na: b\n")
     paths = PathsConfig.force_collection(str(tmp_path))
@@ -346,23 +365,195 @@ def test_collection_details(tmp_path):
     assert details.get_flatmap() is False
 
 
-def test_config_loading_bad_output_format(ansible_config_path):
-    ansible_config_path.write_text("changes_format: combined\noutput_formats: [foobar]")
+def test_config_loading_bad_output_format(collection_config_path):
+    collection_config_path.write_text("changes_format: combined\noutput_formats: 42")
     paths = PathsConfig.detect()
     collection_details = CollectionDetails(paths)
     with pytest.raises(
         ChangelogError,
-        match="Found unknown extension in output_formats: Unknown extension 'foobar'",
+        match="The config value output_formats must be a list",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text("changes_format: combined\noutput_formats:\n- 42")
+    with pytest.raises(
+        ChangelogError,
+        match="The 1st entry of config value output_formats is not a string",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\noutput_formats: [rst, rst]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The output format 'rst' appears more than once",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\noutput_formats: [foobar]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The 1st entry of config value output_formats is an unknown extension: Unknown extension 'foobar'",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+
+def test_config_loading_bad_vcs(collection_config_path):
+    collection_config_path.write_text("changes_format: combined\nvcs: foobar")
+    paths = PathsConfig.detect()
+    collection_details = CollectionDetails(paths)
+    with pytest.raises(
+        ChangelogError,
+        match="Input should be 'none', 'auto' or 'git'",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+
+def test_config_loading_always_refresh(ansible_config_path):
+    ansible_config_path.write_text(
+        "changes_format: combined\nalways_refresh: plugins, fragments"
+    )
+    paths = PathsConfig.detect()
+    collection_details = CollectionDetails(paths)
+    config = ChangelogConfig.load(paths, collection_details)
+    config.always_refresh == "plugins, fragments"
+
+    ansible_config_path.write_text("changes_format: combined\nalways_refresh: 42")
+    with pytest.raises(
+        ChangelogError,
+        match="If specified, always_refresh must be a boolean or a string",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    ansible_config_path.write_text(
+        "changes_format: combined\nalways_refresh: plugins, bar"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match='The config value always_refresh contains an invalid value "bar"',
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    ansible_config_path.write_text(
+        "changes_format: combined\nalways_refresh: plugins, full"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match='The config value always_refresh must not contain "full" together with other values',
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+
+def test_config_loading_bad_sections(collection_config_path):
+    collection_config_path.write_text("changes_format: combined\nsections: 42")
+    paths = PathsConfig.detect()
+    collection_details = CollectionDetails(paths)
+    with pytest.raises(
+        ChangelogError,
+        match="The config value sections must be a list",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text("changes_format: combined\nsections:\n  42: foo")
+    with pytest.raises(
+        ChangelogError,
+        match="The config value sections must be a dictionary mapping strings to strings",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text("changes_format: combined\nsections:\n  foo: 42")
+    with pytest.raises(
+        ChangelogError,
+        match="The config value sections must be a dictionary mapping strings to strings",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [foo, bar]\n- foo"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The 2nd entry of config value sections must be a list of length 2",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [foo, bar]\n- [baz, bam]\n- [foo]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The 3rd entry of config value sections must be a list of length 2",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [foo, bar]\n- [baz, bam]\n- [bam, foo]\n- [1, foo]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The 4th entry of config value sections does not have a string as the first element",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [foo, bar]\n- [baz, bam]\n- [bam, foo]\n- [foo, 1]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The 4th entry of config value sections does not have a string as the second element",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [foo, bar]\n- [foo, baz]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="The section name 'foo' appears more than once",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nsections:\n- [release_summary, foo]"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match=re.escape("No section name must equal prelude_name ('release_summary')"),
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+
+def test_config_loading_collection(collection_config_path):
+    collection_config_path.write_text(
+        "changes_format: combined\nuse_semantic_versioning: false"
+    )
+    paths = PathsConfig.detect()
+    collection_details = CollectionDetails(paths)
+    with pytest.raises(
+        ChangelogError,
+        match="The config value use_semantic_versioning must be true for collections",
+    ):
+        ChangelogConfig.load(paths, collection_details)
+
+    collection_config_path.write_text(
+        "changes_format: combined\nis_other_project: true"
+    )
+    with pytest.raises(
+        ChangelogError,
+        match="is_other_project must not be true for collections",
     ):
         ChangelogConfig.load(paths, collection_details)
 
 
-def test_config_loading_bad_vcs(ansible_config_path):
-    ansible_config_path.write_text("changes_format: combined\nvcs: foobar")
+def test_config_loading_bad_yaml(collection_config_path):
+    collection_config_path.write_text("[]")
     paths = PathsConfig.detect()
     collection_details = CollectionDetails(paths)
     with pytest.raises(
         ChangelogError,
-        match="Invalid VCS value 'foobar'. Must be one of none, git, and auto.",
+        match=" must be a dictionary",
     ):
         ChangelogConfig.load(paths, collection_details)