Repository: useblocks/sphinx-needs
Pull request #1318 (2024-10-07 17:04:29)

♻️  Extract `generate_need` from `add_need` & consolidate warnings
This PR makes the code cleaner and easier to understand for creating needs and handling invalid needs and other build issues, by:

- Extracting `generate_need`: This generates a validated single need dictionary, without adding the need to the build or parsing the content, and then `add_need` calls this.
  Invalid input data results in a `InvalidNeedException`, which replaces/consolidates all the exception/warnings previously raised/emitted.
   Unused exceptions are removed: `NeedsNoIdException`, `NeedsStatusNotAllowed`, `NeedsTagNotAllowed`, `NeedsConstraintNotAllowed`, `NeedsInvalidOption`, `NeedsTemplateException`
- Catch `InvalidNeedException` exceptions in the need/needimport directives and external need loading code, and emit specific warnings, with correct location mapping
- Ensure all warnings have subtypes with certain names and list these in the documentation, explaining how to fail fast and suppress warnings: https://sphinx-needs--1318.org.readthedocs.build/en/1318/configuration.html#build-warnings

---

This activity also surfaced an issue with need creation:
If the content is not parsed, then `parts` and `arch` need fields will not be populated.
- For importing and loading of external needs, this has been fixed by no longer dropping these keys and allowing them to be passed to `add_need`
- For needs from directives which have been set to `hide`

---

There is also a question mark what it means to import a need with nested need directives in the content;
when the content is parsed it will, currently, process these as if they were normal directives in the project,
which will duplicate the needs (since they will likely already be in the imported need set) and ignore any modification on the `needimport`, like `id_prefix`

Essentially, whilst parsing imported need content, should any need (or needimport) directives be ignored? (this could be possible)

(IMHO it's maybe even best to have a "white-list" of allowed directives (and maybe also roles) parsed for imported content, since a lot of these may break the build)

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) Report
Attention: Patch coverage is `82.96296%` with `23 lines` in your changes missing coverage. Please review.
> Project coverage is 87.92%. Comparing base [(`4e10030`)](https://app.codecov.io/gh/useblocks/sphinx-needs/commit/4e10030581c03c59764e517ab750d3ddbb2695d8?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) to head [(`7371a0c`)](https://app.codecov.io/gh/useblocks/sphinx-needs/commit/7371a0c8e817c8f7cd30ec25a9f2477478dd11e7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks).
> Report is 67 commits behind head on master.

| [Files with missing lines](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) | Patch % | Lines |
|---|---|---|
| [sphinx\_needs/api/need.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fapi%2Fneed.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL2FwaS9uZWVkLnB5) | 81.01% | [15 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/directives/needimport.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fdirectives%2Fneedimport.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL2RpcmVjdGl2ZXMvbmVlZGltcG9ydC5weQ==) | 71.42% | [2 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/directives/needservice.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fdirectives%2Fneedservice.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL2RpcmVjdGl2ZXMvbmVlZHNlcnZpY2UucHk=) | 66.66% | [2 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/api/exceptions.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fapi%2Fexceptions.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL2FwaS9leGNlcHRpb25zLnB5) | 92.30% | [1 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/need\_constraints.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fneed_constraints.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL25lZWRfY29uc3RyYWludHMucHk=) | 50.00% | [1 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/needsfile.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Fneedsfile.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL25lZWRzZmlsZS5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |
| [sphinx\_needs/roles/need\_ref.py](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&filepath=sphinx_needs%2Froles%2Fneed_ref.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#diff-c3BoaW54X25lZWRzL3JvbGVzL25lZWRfcmVmLnB5) | 50.00% | [1 Missing :warning: ](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1318      +/-   ##
==========================================
+ Coverage   86.87%   87.92%   +1.04%     
==========================================
  Files          56       60       +4     
  Lines        6532     7013     +481     
==========================================
+ Hits         5675     6166     +491     
+ Misses        857      847      -10     
```

| [Flag](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) | Coverage Δ | |
|---|---|---|
| [pytests](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks) | `87.92% <82.96%> (+1.04%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/useblocks/sphinx-needs/pull/1318?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=useblocks).


Patch:
diff --git a/docs/api.rst b/docs/api.rst
index 75ace83c..d331941e 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -38,7 +38,7 @@ Data
 ----
 
 .. automodule:: sphinx_needs.data
-   :members: NeedsInfoType, NeedsMutable
+   :members: NeedsInfoType, NeedsMutable, NeedsPartType
 
 Views
 -----
diff --git a/docs/conf.py b/docs/conf.py
index 2750ba99..82d81816 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -49,6 +49,7 @@ nitpick_ignore = [
     ("py:class", "docutils.statemachine.StringList"),
     ("py:class", "sphinx_needs.debug.T"),
     ("py:class", "sphinx_needs.views._LazyIndexes"),
+    ("py:class", "sphinx_needs.config.NeedsSphinxConfig"),
 ]
 
 rst_epilog = """
@@ -643,6 +644,7 @@ needs_render_context = {
 
 # build needs.json to make permalinks work
 needs_build_json = True
+needs_reproducible_json = True
 needs_json_remove_defaults = True
 
 # build needs_json for every needs-id to make detail panel
@@ -699,18 +701,36 @@ DEFAULT_DIAGRAM_TEMPLATE = "<size:12>{{type_name}}</size>\\n**{{title|wordwrap(1
 # needs_report_template = "/needs_templates/report_template.need"   # Use custom report template
 
 # -- custom extensions ---------------------------------------
+from typing import get_args  # noqa: E402
 
 from docutils import nodes  # noqa: E402
+from docutils.statemachine import StringList  # noqa: E402
 from sphinx.application import Sphinx  # noqa: E402
 from sphinx.directives import SphinxDirective  # noqa: E402
 from sphinx.roles import SphinxRole  # noqa: E402
 
-from sphinx_needs.api.need import add_external_need  # noqa: E402
+from sphinx_needs.api import generate_need  # noqa: E402
 from sphinx_needs.config import NeedsSphinxConfig  # noqa: E402
-from sphinx_needs.data import SphinxNeedsData  # noqa: E402
+from sphinx_needs.logging import (  # noqa: E402
+    WarningSubTypeDescription,
+    WarningSubTypes,
+)
 from sphinx_needs.needsfile import NeedsList  # noqa: E402
 
 
+class NeedsWarningsDirective(SphinxDirective):
+    """Directive to list all extension warning subtypes."""
+
+    def run(self):
+        parsed = nodes.container(classes=["needs-warnings"])
+        content = [
+            f"- ``needs.{name}`` {WarningSubTypeDescription.get(name, '')}"
+            for name in get_args(WarningSubTypes)
+        ]
+        self.state.nested_parse(StringList(content), self.content_offset, parsed)
+        return [parsed]
+
+
 class NeedExampleDirective(SphinxDirective):
     """Directive to add example content to the documentation.
 
@@ -757,30 +777,26 @@ def create_tutorial_needs(app: Sphinx, _env, _docnames):
 
     We do this dynamically, to avoid having to maintain the JSON file manually.
     """
-    all_data = SphinxNeedsData(app.env).get_needs_mutable()
+    needs_config = NeedsSphinxConfig(app.config)
     writer = NeedsList(app.config, outdir=app.confdir, confdir=app.confdir)
     for i in range(1, 5):
-        test_id = f"T_00{i}"
-        # TODO really here we don't want to create the data, without actually adding it to the needs
-        if test_id in all_data:
-            data = all_data[test_id]
-        else:
-            add_external_need(
-                app,
-                "tutorial-test",
-                id=test_id,
-                title=f"Unit test {i}",
-                content=f"Test case {i}",
-            )
-            data = all_data.pop(test_id)
-        writer.add_need(version, data)
-    # TODO ideally we would only write this file if it has changed (also needimport should add dependency on file)
-    writer.write_json(
-        needs_file="tutorial_needs.json", needs_path=str(Path(app.confdir, "_static"))
-    )
+        need_item = generate_need(
+            needs_config,
+            need_type="tutorial-test",
+            id=f"T_00{i}",
+            title=f"Unit test {i}",
+            content=f"Test case {i}",
+        )
+        writer.add_need(version, need_item)
+    json_str = writer.dump_json()
+    outpath = Path(app.confdir, "_static", "tutorial_needs.json")
+    if outpath.is_file() and outpath.read_text() == json_str:
+        return  # only write this file if it has changed
+    outpath.write_text(json_str)
 
 
 def setup(app: Sphinx):
     app.add_directive("need-example", NeedExampleDirective)
+    app.add_directive("need-warnings", NeedsWarningsDirective)
     app.add_role("need_config_default", NeedConfigDefaultRole())
     app.connect("env-before-read-docs", create_tutorial_needs, priority=600)
diff --git a/docs/configuration.rst b/docs/configuration.rst
index 4957108a..251c164d 100644
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@@ -14,6 +14,24 @@ Add **sphinx_needs** to your extensions.
 
    extensions = ["sphinx_needs",]
 
+.. _config-warnings:
+
+Build Warnings
+--------------
+
+sphinx-needs is designed to be durable and only except when absolutely necessary.
+Any non-fatal issues during the build are logged as Sphinx warnings.
+
+If you wish to "fail-fast" during a build, see the `--fail-on-warning <https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-W>`__ command-line option
+(in sphinx 8.1 ``--exception-on-warning``).
+
+You can also use this in conjunction with the `suppress_warnings <https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-suppress_warnings>`__ configuration option,
+to suppress specific warnings.
+
+Find below a list of all warnings, which can be suppressed:
+
+.. need-warnings::
+
 .. _inc_build:
 
 Incremental build support
@@ -855,8 +873,6 @@ So no ID is autogenerated any more, if this option is set to True:
 
 By default this option is set to **False**.
 
-If an ID is missing Sphinx throws the exception "NeedsNoIdException" and stops the build.
-
 **Example**:
 
 .. code-block:: rst
diff --git a/sphinx_needs/api/__init__.py b/sphinx_needs/api/__init__.py
index 1601cf1d..e4a94382 100644
--- a/sphinx_needs/api/__init__.py
+++ b/sphinx_needs/api/__init__.py
@@ -4,16 +4,18 @@ from .configuration import (
     add_need_type,
     get_need_types,
 )
-from .need import add_external_need, add_need, del_need, get_needs_view, make_hashed_id
+from .exceptions import InvalidNeedException
+from .need import add_external_need, add_need, del_need, generate_need, get_needs_view
 
 __all__ = (
     "add_dynamic_function",
     "add_extra_option",
     "add_external_need",
     "add_need",
+    "InvalidNeedException",
     "add_need_type",
     "del_need",
+    "generate_need",
     "get_need_types",
     "get_needs_view",
-    "make_hashed_id",
 )
diff --git a/sphinx_needs/api/exceptions.py b/sphinx_needs/api/exceptions.py
index 406e1e50..0174a76a 100644
--- a/sphinx_needs/api/exceptions.py
+++ b/sphinx_needs/api/exceptions.py
@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+from typing import Literal
+
 from sphinx.errors import SphinxError, SphinxWarning
 
 
@@ -11,19 +13,54 @@ class NeedsApiConfigException(SphinxError):
     """
 
 
-class NeedsApiConfigWarning(SphinxWarning):
-    pass
-
-
-class NeedsNoIdException(SphinxError):
-    pass
-
+class InvalidNeedException(Exception):
+    """Raised when a need could not be created/added, due to a validation issue."""
+
+    def __init__(
+        self,
+        type_: Literal[
+            "invalid_kwargs",
+            "invalid_type",
+            "missing_id",
+            "invalid_id",
+            "duplicate_id",
+            "invalid_status",
+            "invalid_tags",
+            "invalid_constraints",
+            "invalid_jinja_content",
+            "invalid_template",
+            "global_option",
+        ],
+        message: str,
+    ) -> None:
+        self._type = type_
+        self._message = message
+        super().__init__(f"{message} [{type_}]")
+
+    @property
+    def type(
+        self,
+    ) -> Literal[
+        "invalid_kwargs",
+        "invalid_type",
+        "missing_id",
+        "invalid_id",
+        "duplicate_id",
+        "invalid_status",
+        "invalid_tags",
+        "invalid_constraints",
+        "invalid_jinja_content",
+        "invalid_template",
+        "global_option",
+    ]:
+        return self._type
+
+    @property
+    def message(self) -> str:
+        return self._message
 
-class NeedsStatusNotAllowed(SphinxError):
-    pass
 
-
-class NeedsTagNotAllowed(SphinxError):
+class NeedsApiConfigWarning(SphinxWarning):
     pass
 
 
@@ -31,21 +68,9 @@ class NeedsInvalidException(SphinxError):
     pass
 
 
-class NeedsConstraintNotAllowed(SphinxError):
-    pass
-
-
 class NeedsConstraintFailed(SphinxError):
     pass
 
 
-class NeedsInvalidOption(SphinxError):
-    pass
-
-
-class NeedsTemplateException(SphinxError):
-    pass
-
-
 class NeedsInvalidFilter(SphinxError):
     pass
diff --git a/sphinx_needs/api/need.py b/sphinx_needs/api/need.py
index 34c104cf..8a131721 100644
--- a/sphinx_needs/api/need.py
+++ b/sphinx_needs/api/need.py
@@ -5,7 +5,8 @@ import os
 import re
 import warnings
 from contextlib import contextmanager
-from typing import Any, Iterator
+from pathlib import Path
+from typing import Any, Iterable, Iterator
 
 from docutils import nodes
 from docutils.parsers.rst.states import RSTState
@@ -14,17 +15,9 @@ from jinja2 import Template
 from sphinx.application import Sphinx
 from sphinx.environment import BuildEnvironment
 
-from sphinx_needs.api.exceptions import (
-    NeedsConstraintNotAllowed,
-    NeedsInvalidException,
-    NeedsInvalidOption,
-    NeedsNoIdException,
-    NeedsStatusNotAllowed,
-    NeedsTagNotAllowed,
-    NeedsTemplateException,
-)
+from sphinx_needs.api.exceptions import InvalidNeedException
 from sphinx_needs.config import NEEDS_CONFIG, GlobalOptionsType, NeedsSphinxConfig
-from sphinx_needs.data import NeedsInfoType, SphinxNeedsData
+from sphinx_needs.data import NeedsInfoType, NeedsPartType, SphinxNeedsData
 from sphinx_needs.directives.needuml import Needuml, NeedumlException
 from sphinx_needs.filter_common import filter_single_need
 from sphinx_needs.logging import get_logger, log_warning
@@ -38,21 +31,22 @@ logger = get_logger(__name__)
 _deprecated_kwargs = {"constraints_passed", "links_string", "hide_tags", "hide_status"}
 
 
-def add_need(
-    app: Sphinx,
-    state: None | RSTState,
-    docname: None | str,
-    lineno: None | int,
+def generate_need(
+    needs_config: NeedsSphinxConfig,
     need_type: str,
     title: str,
     *,
+    docname: None | str = None,
+    lineno: None | int = None,
     id: str | None = None,
-    content: str | StringList = "",
+    doctype: str = ".rst",
+    content: str = "",
     lineno_content: None | int = None,
-    doctype: None | str = None,
     status: str | None = None,
     tags: None | str | list[str] = None,
     constraints: None | str | list[str] = None,
+    parts: dict[str, NeedsPartType] | None = None,
+    arch: dict[str, str] | None = None,
     signature: str = "",
     sections: list[str] | None = None,
     delete: None | bool = False,
@@ -61,55 +55,28 @@ def add_need(
     collapse: None | bool = None,
     style: None | str = None,
     layout: None | str = None,
+    template_root: Path | None = None,
     template: None | str = None,
     pre_template: str | None = None,
     post_template: str | None = None,
     is_external: bool = False,
     external_url: str | None = None,
     external_css: str = "external_link",
-    **kwargs: Any,
-) -> list[nodes.Node]:
-    """
-    Creates a new need and returns its node.
-
-    ``add_need`` allows to create needs programmatically and use its returned node to be integrated in any
-    docutils based structure.
-
-    ``kwags`` can contain options defined in ``needs_extra_options`` and ``needs_extra_links``.
-    If an entry is found in ``kwags``, which *is not* specified in the configuration or registered e.g. via
-    ``add_extra_option``, an exception is raised.
-
-    If ``is_external`` is set to ``True``, no node will be created.
-    Instead, the need is referencing an external url.
-    Used mostly for :ref:`needs_external_needs` to integrate and reference needs from external documentation.
-
-    **Usage**:
-
-    Normally needs get created during handling of a specialised directive.
-    So this pseudocode shows how to use ``add_need`` inside such a directive.
-
-    .. code-block:: python
+    **kwargs: str,
+) -> NeedsInfoType:
+    """Creates a validated need data entry, without adding it to the project.
 
-        from sphinx.util.docutils import SphinxDirective
-        from sphinx_needs.api import add_need
+    .. important:: This function does not parse or analyse the content,
+        and so will not auto-populate the ``parts`` or ``arch`` fields of the need
+        from the content.
 
-        class MyDirective(SphinxDirective)
-            # configs and init routine
+        It will also not validate that the ID is not already in use within the project.
 
-            def run():
-                main_section = []
+    :raises InvalidNeedException: If the data fails any validation issue.
 
-                docname = self.env.docname
-
-                # All needed sphinx-internal information we can take from our current directive class.
-                # e..g app, state, lineno
-                main_section += add_need(self.env.app, self.state, docname, self.lineno,
-                                         need_type="req", title="my title", id="ID_001"
-                                         content=self.content)
-
-                # Feel free to add custom stuff to main_section like sections, text, ...
-
-                return main_section
+    ``kwargs`` can contain options defined in ``needs_extra_options`` and ``needs_extra_links``.
+    If an entry is found in ``kwargs``, which *is not* specified in the configuration or registered e.g. via
+    ``add_extra_option``, an exception is raised.
 
     If the need is within the current project, i.e. not an external need,
     the following parameters are used to help provide source mapped warnings and errors:
@@ -131,8 +98,7 @@ def add_need(
     :param need_type: Name of the need type to create.
     :param title: String as title.
     :param id: ID as string. If not given, an id will get generated.
-    :param content: Content of the need, either as a ``str``
-        or a ``StringList`` (a string with mapping to the source text).
+    :param content: Content of the need
     :param status: Status as string.
     :param tags: A list of tags, or a comma separated string.
     :param constraints: Constraints as single, comma separated, string.
@@ -142,144 +108,74 @@ def add_need(
     :param collapse: boolean value.
     :param style: String value of class attribute of node.
     :param layout: String value of layout definition to use
+    :param template_root: Root path for template files, only required if the template_path config is relative.
     :param template: Template name to use for the content of this need
     :param pre_template: Template name to use for content added before need
     :param post_template: Template name to use for the content added after need
-
-    :return: node
     """
-
-    if kwargs.keys() & _deprecated_kwargs:
-        warnings.warn(
-            "deprecated key found in kwargs", DeprecationWarning, stacklevel=1
-        )
-        kwargs = {k: v for k, v in kwargs.items() if k not in _deprecated_kwargs}
-
-    #############################################################################################
-    # Get environment
-    #############################################################################################
-    env = app.env
-    needs_config = NeedsSphinxConfig(app.config)
-    types = needs_config.types
-    type_name = ""
-    type_prefix = ""
-    type_color = ""
-    type_style = ""
-    found = False
-
     # location is used to provide source mapped warnings
     location = (docname, lineno) if docname else None
 
-    # Log messages for need elements that could not be imported.
-    configured_need_types = [ntype["directive"] for ntype in types]
-    if need_type not in configured_need_types:
-        log_warning(
-            logger,
-            f"Couldn't create need {id}. Reason: The need-type (i.e. `{need_type}`) is not set "
-            "in the project's 'need_types' configuration in conf.py.",
-            "add",
-            location=location,
+    # validate kwargs
+    allowed_kwargs = {x["option"] for x in needs_config.extra_links} | set(
+        NEEDS_CONFIG.extra_options
+    )
+    unknown_kwargs = set(kwargs) - allowed_kwargs
+    if unknown_kwargs:
+        raise InvalidNeedException(
+            "invalid_kwargs",
+            f"Options {unknown_kwargs!r} not in 'needs_extra_options' or 'needs_extra_links.",
         )
 
-    for ntype in types:
-        if ntype["directive"] == need_type:
-            type_name = ntype["title"]
-            type_prefix = ntype["prefix"]
-            type_color = ntype.get("color") or "#000000"
-            type_style = ntype.get("style") or "node"
-            found = True
-            break
+    # get the need type data
+    configured_need_types = {ntype["directive"]: ntype for ntype in needs_config.types}
+    if not (need_type_data := configured_need_types.get(need_type)):
+        raise InvalidNeedException("invalid_type", f"Unknown need type {need_type!r}.")
 
-    if delete:
-        # Don't generate a need object if the :delete: option is enabled.
-        return [nodes.Text("")]
-    if not found:
-        # This should never happen. But it may happen, if Sphinx is called multiples times
-        # inside one ongoing python process.
-        # In this case the configuration from a prior sphinx run may be active, which has registered a directive,
-        # which is reused inside a current document, but no type was defined for the current run...
-        # Yeah, this really has happened...
-        return [nodes.Text("")]
-
-    # Get the id or generate a random string/hash string, which is hopefully unique
-    # TODO: Check, if id was already given. If True, recalculate id
-    # id = self.options.get("id", ''.join(random.SystemRandom().choice(string.ascii_uppercase + string.digits) for
-    # _ in range(5)))
+    # generate and validate the id
     if id is None and needs_config.id_required:
-        raise NeedsNoIdException(
-            "An id is missing for this need and must be set, because 'needs_id_required' "
-            f"is set to True in conf.py. Need '{title}' in {docname} ({lineno})"
+        raise InvalidNeedException(
+            "missing_id", "No ID defined, but 'needs_id_required' is set to True."
         )
-
-    if id is None:
-        need_id = make_hashed_id(
-            app,
-            need_type,
-            title,
-            "\n".join(content) if isinstance(content, StringList) else content,
-        )
-    else:
-        need_id = id
-
+    need_id = (
+        _make_hashed_id(need_type_data["prefix"], title, content, needs_config)
+        if id is None
+        else id
+    )
     if (
         needs_config.id_regex
         and not is_external
         and not re.match(needs_config.id_regex, need_id)
     ):
-        raise NeedsInvalidException(
-            f"Given ID '{need_id}' does not match configured regex '{needs_config.id_regex}'"
+        raise InvalidNeedException(
+            "invalid_id",
+            f"Given ID {need_id!r} does not match configured regex {needs_config.id_regex!r}",
         )
 
-    # Handle status
-    # Check if status is in needs_statuses. If not raise an error.
+    # validate status
     if needs_config.statuses and status not in [
         stat["name"] for stat in needs_config.statuses
     ]:
-        raise NeedsStatusNotAllowed(
-            f"Status {status} of need id {need_id} is not allowed "
-            "by config value 'needs_statuses'."
+        raise InvalidNeedException(
+            "invalid_status", f"Status {status!r} not in 'needs_statuses'."
         )
 
+    # validate tags
     tags = _split_list_with_dyn_funcs(tags, location)
-    # Check if tag is in needs_tags. If not raise an error.
-    if needs_config.tags:
-        needs_tags = [t["name"] for t in needs_config.tags]
-        for tag in tags:
-            if tag not in needs_tags:
-                raise NeedsTagNotAllowed(
-                    f"Tag {tag} of need id {need_id} is not allowed "
-                    "by config value 'needs_tags'."
-                )
+    if needs_config.tags and (
+        unknown_tags := set(tags) - {t["name"] for t in needs_config.tags}
+    ):
+        raise InvalidNeedException(
+            "invalid_tags", f"Tags {unknown_tags!r} not in 'needs_tags'."
+        )
 
+    # validate constraints
     constraints = _split_list_with_dyn_funcs(constraints, location)
-    # Check if constraint is in needs_constraints. If not raise an error.
-    if needs_config.constraints:
-        for constraint in constraints:
-            if constraint not in needs_config.constraints:
-                raise NeedsConstraintNotAllowed(
-                    f"Constraint {constraint} of need id {need_id} is not allowed "
-                    "by config value 'needs_constraints'."
-                )
-
-    #############################################################################################
-    # Add need to global need list
-    #############################################################################################
-
-    if SphinxNeedsData(env).has_need(need_id):
-        if id:
-            message = f"A need with ID {need_id} already exists, " f"title: {title!r}."
-        else:  # this is a generated ID
-            _title = " ".join(title)
-            message = (
-                "Needs could not generate a unique ID for a need with "
-                f"the title {_title!r} because another need had the same title. "
-                "Either supply IDs for the requirements or ensure the "
-                "titles are different.  NOTE: If title is being generated "
-                "from the content, then ensure the first sentence of the "
-                "requirements are different."
-            )
-        log_warning(logger, message, "duplicate_id", location=location)
-        return []
+    if unknown_constraints := set(constraints) - set(needs_config.constraints):
+        raise InvalidNeedException(
+            "invalid_constraints",
+            f"Constraints {unknown_constraints!r} not in 'needs_constraints'.",
+        )
 
     # Trim title if it is too long
     max_length = needs_config.max_title_length
@@ -290,22 +186,18 @@ def add_need(
     else:
         trimmed_title = title[: max_length - 3] + "..."
 
-    # Calculate doc type, e.g. .rst or .md
-    if doctype is None:
-        doctype = os.path.splitext(env.doc2path(docname))[1] if docname else ""
-
     # Add the need and all needed information
     needs_info: NeedsInfoType = {
         "docname": docname,
         "lineno": lineno,
         "lineno_content": lineno_content,
         "doctype": doctype,
-        "content": "\n".join(content) if isinstance(content, StringList) else content,
+        "content": content,
         "type": need_type,
-        "type_name": type_name,
-        "type_prefix": type_prefix,
-        "type_color": type_color,
-        "type_style": type_style,
+        "type_name": need_type_data["title"],
+        "type_prefix": need_type_data["prefix"],
+        "type_color": need_type_data.get("color") or "#000000",
+        "type_style": need_type_data.get("style") or "node",
         "status": status,
         "tags": tags,
         "constraints": constraints,
@@ -315,7 +207,7 @@ def add_need(
         "title": trimmed_title,
         "full_title": title,
         "collapse": collapse or False,
-        "arch": {},  # extracted later
+        "arch": arch or {},
         "style": style,
         "layout": layout,
         "template": template,
@@ -324,7 +216,7 @@ def add_need(
         "hide": hide,
         "delete": delete or False,
         "jinja_content": jinja_content or False,
-        "parts": {},
+        "parts": parts or {},
         "is_part": False,
         "is_need": True,
         "id_parent": need_id,
@@ -341,33 +233,23 @@ def add_need(
         "signature": signature,
         "parent_need": "",
     }
-    needs_extra_option_names = list(NEEDS_CONFIG.extra_options)
-    _merge_extra_options(needs_info, kwargs, needs_extra_option_names)
-
-    needs_global_options = needs_config.global_options
-    _merge_global_options(app, needs_info, needs_global_options)
-
-    link_names = [x["option"] for x in needs_config.extra_links]
-    for keyword in kwargs:
-        if keyword not in needs_extra_option_names and keyword not in link_names:
-            raise NeedsInvalidOption(
-                f"Unknown Option {keyword}. "
-                "Use needs_extra_options or needs_extra_links in conf.py"
-                "to define this option."
-            )
+
+    # add dynamic keys to needs_info
+    _merge_extra_options(needs_info, kwargs, NEEDS_CONFIG.extra_options)
+    _merge_global_options(needs_config, needs_info, needs_config.global_options)
 
     # Merge links
-    copy_links = []
+    copy_links: list[str] = []
 
     for link_type in needs_config.extra_links:
         # Check, if specific link-type got some arguments during method call
         if (
             link_type["option"] not in kwargs
-            and link_type["option"] not in needs_global_options
+            and link_type["option"] not in needs_config.global_options
         ):
             # if not we set no links, but entry in needS_info must be there
             links = []
-        elif link_type["option"] in needs_global_options and (
+        elif link_type["option"] in needs_config.global_options and (
             link_type["option"] not in kwargs
             or len(str(kwargs[link_type["option"]])) == 0
         ):
@@ -382,10 +264,7 @@ def add_need(
         needs_info[link_type["option"]] = links
         needs_info["{}_back".format(link_type["option"])] = []
 
-        if "copy" not in link_type:
-            link_type["copy"] = False
-
-        if link_type["copy"] and link_type["option"] != "links":
+        if link_type.get("copy", False) and link_type["option"] != "links":
             copy_links += links  # Save extra links for main-links
 
     needs_info["links"] += copy_links  # Set copied links to main-links
@@ -398,27 +277,188 @@ def add_need(
         need_content_context = {**needs_info}
         need_content_context.update(**needs_config.filter_data)
         need_content_context.update(**needs_config.render_context)
-        needs_info["content"] = content = jinja_parse(
-            need_content_context, needs_info["content"]
-        )
+        try:
+            needs_info["content"] = jinja_parse(
+                need_content_context, needs_info["content"]
+            )
+        except Exception as e:
+            raise InvalidNeedException(
+                "invalid_jinja_content",
+                f"Error while rendering content: {e}",
+            )
 
     if needs_info["template"]:
-        needs_info["content"] = content = _prepare_template(app, needs_info, "template")
+        needs_info["content"] = _prepare_template(
+            needs_config, needs_info, "template", template_root
+        )
 
     if needs_info["pre_template"]:
-        needs_info["pre_content"] = _prepare_template(app, needs_info, "pre_template")
+        needs_info["pre_content"] = _prepare_template(
+            needs_config, needs_info, "pre_template", template_root
+        )
 
     if needs_info["post_template"]:
-        needs_info["post_content"] = _prepare_template(app, needs_info, "post_template")
+        needs_info["post_content"] = _prepare_template(
+            needs_config, needs_info, "post_template", template_root
+        )
+
+    return needs_info
+
+
+def add_need(
+    app: Sphinx,
+    state: None | RSTState,
+    docname: None | str,
+    lineno: None | int,
+    need_type: str,
+    title: str,
+    *,
+    id: str | None = None,
+    content: str | StringList = "",
+    lineno_content: None | int = None,
+    doctype: None | str = None,
+    status: str | None = None,
+    tags: None | str | list[str] = None,
+    constraints: None | str | list[str] = None,
+    parts: dict[str, NeedsPartType] | None = None,
+    arch: dict[str, str] | None = None,
+    signature: str = "",
+    sections: list[str] | None = None,
+    delete: None | bool = False,
+    jinja_content: None | bool = False,
+    hide: bool = False,
+    collapse: None | bool = None,
+    style: None | str = None,
+    layout: None | str = None,
+    template: None | str = None,
+    pre_template: str | None = None,
+    post_template: str | None = None,
+    is_external: bool = False,
+    external_url: str | None = None,
+    external_css: str = "external_link",
+    **kwargs: Any,
+) -> list[nodes.Node]:
+    """
+    Creates a new need and returns its node.
+
+    ``add_need`` allows to create needs programmatically and use its returned node to be integrated in any
+    docutils based structure.
+
+    ``kwargs`` can contain options defined in ``needs_extra_options`` and ``needs_extra_links``.
+    If an entry is found in ``kwargs``, which *is not* specified in the configuration or registered e.g. via
+    ``add_extra_option``, an exception is raised.
+
+    If ``is_external`` is set to ``True``, no node will be created.
+    Instead, the need is referencing an external url.
+    Used mostly for :ref:`needs_external_needs` to integrate and reference needs from external documentation.
+
+    :raises InvalidNeedException: If the need could not be added due to a validation issue.
+
+    If the need is within the current project, i.e. not an external need,
+    the following parameters are used to help provide source mapped warnings and errors:
+
+    :param docname: documentation identifier, for the referencing document.
+    :param lineno: line number of the top of the directive (1-indexed).
+    :param lineno_content: line number of the content start of the directive (1-indexed).
+
+    Otherwise, the following parameters are used:
+
+    :param is_external: Is true, no node is created and need is referencing external url
+    :param external_url: URL as string, which is used as target if ``is_external`` is ``True``
+    :param external_css: CSS class name as string, which is set for the <a> tag.
+
+    Additional parameters:
+
+    :param app: Sphinx application object.
+    :param state: Current state object.
+    :param need_type: Name of the need type to create.
+    :param title: String as title.
+    :param id: ID as string. If not given, an id will get generated.
+    :param content: Content of the need, either as a ``str``
+        or a ``StringList`` (a string with mapping to the source text).
+    :param status: Status as string.
+    :param tags: A list of tags, or a comma separated string.
+    :param constraints: Constraints as single, comma separated, string.
+    :param constraints_passed: Contains bool describing if all constraints have passed
+    :param delete: boolean value (Remove the complete need).
+    :param hide: boolean value.
+    :param collapse: boolean value.
+    :param style: String value of class attribute of node.
+    :param layout: String value of layout definition to use
+    :param template: Template name to use for the content of this need
+    :param pre_template: Template name to use for content added before need
+    :param post_template: Template name to use for the content added after need
+
+    :return: list of nodes
+    """
+    # remove deprecated kwargs
+    if kwargs.keys() & _deprecated_kwargs:
+        warnings.warn(
+            "deprecated key found in kwargs", DeprecationWarning, stacklevel=1
+        )
+        kwargs = {k: v for k, v in kwargs.items() if k not in _deprecated_kwargs}
 
-    SphinxNeedsData(env).add_need(needs_info)
+    if delete:
+        # Don't generate a need object if the :delete: option is enabled.
+        return []
+
+    if doctype is None and not is_external and docname:
+        doctype = os.path.splitext(app.env.doc2path(docname))[1]
+
+    needs_info = generate_need(
+        needs_config=NeedsSphinxConfig(app.config),
+        need_type=need_type,
+        title=title,
+        docname=docname,
+        lineno=lineno,
+        id=id,
+        doctype=doctype or "",
+        content="\n".join(content) if isinstance(content, StringList) else content,
+        lineno_content=lineno_content,
+        status=status,
+        tags=tags,
+        constraints=constraints,
+        parts=parts,
+        arch=arch,
+        signature=signature,
+        sections=sections,
+        delete=delete,
+        jinja_content=jinja_content,
+        hide=hide,
+        collapse=collapse,
+        style=style,
+        layout=layout,
+        template_root=Path(str(app.srcdir)),
+        template=template,
+        pre_template=pre_template,
+        post_template=post_template,
+        is_external=is_external,
+        external_url=external_url,
+        external_css=external_css,
+        **kwargs,
+    )
+
+    if SphinxNeedsData(app.env).has_need(needs_info["id"]):
+        if id is None:
+            # this is a generated ID
+            message = f"Unique ID could not be generated for need with title {needs_info['full_title']!r}."
+        else:
+            message = f"A need with ID {needs_info['id']!r} already exists."
+        raise InvalidNeedException("duplicate_id", message)
+
+    SphinxNeedsData(app.env).add_need(needs_info)
 
     if needs_info["is_external"]:
         return []
 
     assert state is not None, "parser state must be set if need is not external"
 
-    return _create_need_node(needs_info, env, state, content)
+    if needs_info["jinja_content"] or needs_info["template"]:
+        # if the content was generated by jinja,
+        # then we can no longer use the original potentially source mapped StringList
+        content = needs_info["content"]
+
+    return _create_need_node(needs_info, app.env, state, content)
 
 
 @contextmanager
@@ -466,6 +506,8 @@ def _create_need_node(
     if data["hide"]:
         # still add node to doctree, so we can later compute its relative location in the document
         # (see analyse_need_locations function)
+        # TODO this is problematic because it will not populate ``parts`` or ``arch`` of the need,
+        # nor will it find/add any child needs
         node_need["hidden"] = True
         return [node_need]
 
@@ -503,6 +545,7 @@ def _create_need_node(
         )
 
     # Extract plantuml diagrams and store needumls with keys in arch, e.g. need_info['arch']['diagram']
+    data["arch"] = {}
     node_need_needumls_without_key = []
     node_need_needumls_key_names = []
     for child in node_need.children:
@@ -529,6 +572,7 @@ def _create_need_node(
     if node_need_needumls_without_key:
         data["arch"]["diagram"] = node_need_needumls_without_key[0]["content"]
 
+    data["parts"] = {}
     need_parts = find_parts(node_need)
     update_need_with_parts(env, data, need_parts)
 
@@ -559,7 +603,7 @@ def del_need(app: Sphinx, need_id: str) -> None:
     """
     data = SphinxNeedsData(app.env)
     if not data.has_need(need_id):
-        log_warning(logger, f"Given need id {need_id} not exists!", None, None)
+        log_warning(logger, f"Given need id {need_id} not exists!", "delete_need", None)
     else:
         data.remove_need(need_id)
 
@@ -596,6 +640,8 @@ def add_external_need(
     :param constraints: constraints as single, comma separated string.
     :param external_css: CSS class name as string, which is set for the <a> tag.
 
+    :raises InvalidNeedException: If the need could not be added due to a validation issue.
+
     """
     for fixed_key in ("state", "docname", "lineno", "is_external"):
         if fixed_key in kwargs:
@@ -627,73 +673,56 @@ def add_external_need(
     )
 
 
-def _prepare_template(app: Sphinx, needs_info: NeedsInfoType, template_key: str) -> str:
-    needs_config = NeedsSphinxConfig(app.config)
-    template_folder = needs_config.template_folder
-    if not os.path.isabs(template_folder):
-        template_folder = os.path.join(app.srcdir, template_folder)
+def _prepare_template(
+    needs_config: NeedsSphinxConfig,
+    needs_info: NeedsInfoType,
+    template_key: str,
+    template_root: None | Path,
+) -> str:
+    template_folder = Path(needs_config.template_folder)
+    if not template_folder.is_absolute():
+        if template_root is None:
+            raise InvalidNeedException(
+                "invalid_template",
+                "Template folder is not an absolute path and no template_root is given.",
+            )
+        template_folder = template_root / template_folder
 
-    if not os.path.isdir(template_folder):
-        raise NeedsTemplateException(
-            f"Template folder does not exist: {template_folder}"
+    if not template_folder.is_dir():
+        raise InvalidNeedException(
+            "invalid_template", f"Template folder does not exist: {template_folder}"
         )
 
-    template_file_name = needs_info[template_key] + ".need"
-    template_path = os.path.join(template_folder, template_file_name)
-    if not os.path.isfile(template_path):
-        raise NeedsTemplateException(f"Template does not exist: {template_path}")
+    template_file_name = str(needs_info[template_key]) + ".need"
+    template_path = template_folder / template_file_name
+    if not template_path.is_file():
+        raise InvalidNeedException(
+            "invalid_template", f"Template does not exist: {template_path}"
+        )
 
-    with open(template_path) as template_file:
+    with template_path.open() as template_file:
         template_content = "".join(template_file.readlines())
-    template_obj = Template(template_content)
-    new_content = template_obj.render(**needs_info, **needs_config.render_context)
+    try:
+        template_obj = Template(template_content)
+        new_content = template_obj.render(**needs_info, **needs_config.render_context)
+    except Exception as e:
+        raise InvalidNeedException(
+            "invalid_template",
+            f"Error while rendering template {template_path}: {e}",
+        )
 
     return new_content
 
 
-def make_hashed_id(
-    app: Sphinx,
-    need_type: str,
-    full_title: str,
-    content: str,
-    id_length: int | None = None,
+def _make_hashed_id(
+    type_prefix: str, full_title: str, content: str, config: NeedsSphinxConfig
 ) -> str:
-    """
-    Creates an ID based on title or need.
-
-    Also cares about the correct prefix, which is specified for each need type.
-
-    :param app: Sphinx application object
-    :param need_type: name of the need directive, e.g. req
-    :param full_title: full title of the need
-    :param content: content of the need
-    :param id_length: maximum length of the generated ID
-    :return: ID as string
-    """
-    needs_config = NeedsSphinxConfig(app.config)
-    types = needs_config.types
-    if id_length is None:
-        id_length = needs_config.id_length
-    type_prefix = None
-    for ntype in types:
-        if ntype["directive"] == need_type:
-            type_prefix = ntype["prefix"]
-            break
-    if type_prefix is None:
-        raise NeedsInvalidException(
-            f"Given need_type {need_type} is unknown. File {app.env.docname}"
-        )
-
-    hashable_content = full_title or "\n".join(content)
-    hashed_id = hashlib.sha1(hashable_content.encode("UTF-8")).hexdigest().upper()
-
-    # check if needs_id_from_title is configured
-    cal_hashed_id = hashed_id
-    if needs_config.id_from_title:
-        id_from_title = full_title.upper().replace(" ", "_") + "_"
-        cal_hashed_id = id_from_title + hashed_id
-
-    return f"{type_prefix}{cal_hashed_id[:id_length]}"
+    """Create an ID based on the type and title of the need."""
+    hashable_content = full_title or content
+    hashed = hashlib.sha1(hashable_content.encode("UTF-8")).hexdigest().upper()
+    if config.id_from_title:
+        hashed = full_title.upper().replace(" ", "_") + "_" + hashed
+    return f"{type_prefix}{hashed[:config.id_length]}"
 
 
 def _split_list_with_dyn_funcs(
@@ -732,7 +761,7 @@ def _split_list_with_dyn_funcs(
                 log_warning(
                     logger,
                     f"Dynamic function not closed correctly: {text}",
-                    None,
+                    "dynamic_function",
                     location=location,
                 )
             text = text[2:]
@@ -751,7 +780,7 @@ def _split_list_with_dyn_funcs(
 def _merge_extra_options(
     needs_info: NeedsInfoType,
     needs_kwargs: dict[str, Any],
-    needs_extra_options: list[str],
+    needs_extra_options: Iterable[str],
 ) -> set[str]:
     """Add any extra options introduced via options_ext to needs_info"""
     extra_keys = set(needs_kwargs.keys()).difference(set(needs_info.keys()))
@@ -768,12 +797,13 @@ def _merge_extra_options(
 
 
 def _merge_global_options(
-    app: Sphinx, needs_info: NeedsInfoType, global_options: GlobalOptionsType
+    config: NeedsSphinxConfig,
+    needs_info: NeedsInfoType,
+    global_options: GlobalOptionsType,
 ) -> None:
     """Add all global defined options to needs_info"""
     if global_options is None:
         return
-    config = NeedsSphinxConfig(app.config)
     for key, value in global_options.items():
         # If key already exists in needs_info, this global_option got overwritten manually in current need
         if needs_info.get(key):
@@ -791,8 +821,9 @@ def _merge_global_options(
             # TODO should first match break loop?
             if len(single_value) < 2 or len(single_value) > 3:
                 # TODO this should be validated earlier at the "config" level
-                raise NeedsInvalidException(
-                    f"global option tuple has wrong amount of parameters: {key}"
+                raise InvalidNeedException(
+                    "global_option",
+                    f"global option tuple has wrong amount of parameters: {key!r}",
                 )
             if filter_single_need(needs_info, config, single_value[1]):
                 # Set value, if filter has matched
diff --git a/sphinx_needs/config.py b/sphinx_needs/config.py
index 0765f751..2cde4a08 100644
--- a/sphinx_needs/config.py
+++ b/sphinx_needs/config.py
@@ -158,7 +158,7 @@ class LinkOptionsType(TypedDict, total=False):
     outgoing: str
     """The outgoing link title"""
     copy: bool
-    """Copy to common links data. Default: True"""
+    """Copy to common links data. Default: False"""
     color: str
     """Used for needflow. Default: #000000"""
     style: str
diff --git a/sphinx_needs/data.py b/sphinx_needs/data.py
index 0fb95fc4..1428e6ba 100644
--- a/sphinx_needs/data.py
+++ b/sphinx_needs/data.py
@@ -149,8 +149,6 @@ NeedsCoreFields: Final[Mapping[str, CoreFieldParameters]] = {
             "additionalProperties": {"type": "string"},
             "default": {},
         },
-        "exclude_external": True,
-        "exclude_import": True,
     },
     "is_external": {
         "description": "If true, no node is created and need is referencing external url.",
@@ -231,8 +229,6 @@ NeedsCoreFields: Final[Mapping[str, CoreFieldParameters]] = {
             "additionalProperties": {"type": "object"},
             "default": {},
         },
-        "exclude_external": True,
-        "exclude_import": True,
     },
     "id_parent": {
         "description": "<parent ID>, or <self ID> if not a part.",
diff --git a/sphinx_needs/directives/need.py b/sphinx_needs/directives/need.py
index 8069c0af..90404b8b 100644
--- a/sphinx_needs/directives/need.py
+++ b/sphinx_needs/directives/need.py
@@ -9,7 +9,7 @@ from sphinx.application import Sphinx
 from sphinx.environment import BuildEnvironment
 from sphinx.util.docutils import SphinxDirective
 
-from sphinx_needs.api import add_need
+from sphinx_needs.api import InvalidNeedException, add_need
 from sphinx_needs.config import NEEDS_CONFIG, NeedsSphinxConfig
 from sphinx_needs.data import NeedsMutable, SphinxNeedsData
 from sphinx_needs.debug import measure_time
@@ -83,30 +83,39 @@ class NeedDirective(SphinxDirective):
         for extra_option in NEEDS_CONFIG.extra_options:
             need_extra_options[extra_option] = self.options.get(extra_option, "")
 
-        need_nodes = add_need(
-            self.env.app,
-            self.state,
-            self.env.docname,
-            self.lineno,
-            need_type=self.name,
-            title=title,
-            id=id,
-            content=self.content,
-            lineno_content=self.content_offset + 1,
-            status=status,
-            tags=tags,
-            hide=hide,
-            template=template,
-            pre_template=pre_template,
-            post_template=post_template,
-            collapse=collapse,
-            style=style,
-            layout=layout,
-            delete=delete_opt,
-            jinja_content=jinja_content,
-            constraints=constraints,
-            **need_extra_options,
-        )
+        try:
+            need_nodes = add_need(
+                self.env.app,
+                self.state,
+                self.env.docname,
+                self.lineno,
+                need_type=self.name,
+                title=title,
+                id=id,
+                content=self.content,
+                lineno_content=self.content_offset + 1,
+                status=status,
+                tags=tags,
+                hide=hide,
+                template=template,
+                pre_template=pre_template,
+                post_template=post_template,
+                collapse=collapse,
+                style=style,
+                layout=layout,
+                delete=delete_opt,
+                jinja_content=jinja_content,
+                constraints=constraints,
+                **need_extra_options,
+            )
+        except InvalidNeedException as err:
+            log_warning(
+                LOGGER,
+                f"Need could not be created: {err.message}",
+                "create_need",
+                location=self.get_location(),
+            )
+            return []
         add_doc(self.env, self.env.docname)
         return need_nodes
 
diff --git a/sphinx_needs/directives/needextend.py b/sphinx_needs/directives/needextend.py
index 37be5f2a..049aa708 100644
--- a/sphinx_needs/directives/needextend.py
+++ b/sphinx_needs/directives/needextend.py
@@ -126,7 +126,7 @@ def extend_needs_data(
                 log_warning(
                     logger,
                     error,
-                    "extend",
+                    "needextend",
                     location=(
                         current_needextend["docname"],
                         current_needextend["lineno"],
@@ -149,7 +149,7 @@ def extend_needs_data(
                 log_warning(
                     logger,
                     f"Invalid filter {need_filter!r}: {e}",
-                    "extend",
+                    "needextend",
                     location=(
                         current_needextend["docname"],
                         current_needextend["lineno"],
@@ -172,7 +172,7 @@ def extend_needs_data(
                                 log_warning(
                                     logger,
                                     f"Provided link id {item} for needextend does not exist.",
-                                    None,
+                                    "needextend",
                                     location=(
                                         current_needextend["docname"],
                                         current_needextend["lineno"],
@@ -207,7 +207,7 @@ def extend_needs_data(
                                 log_warning(
                                     logger,
                                     f"Provided link id {item} for needextend does not exist.",
-                                    None,
+                                    "needextend",
                                     location=(
                                         current_needextend["docname"],
                                         current_needextend["lineno"],
diff --git a/sphinx_needs/directives/needflow/_plantuml.py b/sphinx_needs/directives/needflow/_plantuml.py
index 9d10b137..40f23a8a 100644
--- a/sphinx_needs/directives/needflow/_plantuml.py
+++ b/sphinx_needs/directives/needflow/_plantuml.py
@@ -222,8 +222,8 @@ def process_needflow_plantuml(
                         flow=current_needflow["target_id"],
                         link_types=",".join(link_type_names),
                     ),
-                    None,
-                    None,
+                    "needflow",
+                    location=node,
                 )
 
         # compute the allowed link names
diff --git a/sphinx_needs/directives/needgantt.py b/sphinx_needs/directives/needgantt.py
index 3dc926df..aa51d602 100644
--- a/sphinx_needs/directives/needgantt.py
+++ b/sphinx_needs/directives/needgantt.py
@@ -259,7 +259,7 @@ def process_needgantt(
                         logger,
                         "Duration not set or invalid for needgantt chart. "
                         f"Need: {need['id']!r}{need_location}. Duration: {duration!r}",
-                        "gantt",
+                        "needgantt",
                         location=node,
                     )
                     duration = 1
diff --git a/sphinx_needs/directives/needimport.py b/sphinx_needs/directives/needimport.py
index d216813f..87590ded 100644
--- a/sphinx_needs/directives/needimport.py
+++ b/sphinx_needs/directives/needimport.py
@@ -12,7 +12,7 @@ from docutils.parsers.rst import directives
 from requests_file import FileAdapter
 from sphinx.util.docutils import SphinxDirective
 
-from sphinx_needs.api import add_need
+from sphinx_needs.api import InvalidNeedException, add_need
 from sphinx_needs.config import NEEDS_CONFIG, NeedsSphinxConfig
 from sphinx_needs.data import NeedsCoreFields, NeedsInfoType
 from sphinx_needs.debug import measure_time
@@ -98,7 +98,7 @@ class NeedimportDirective(SphinxDirective):
                             "Deprecation warning: Relative path must be relative to the current document in future, "
                             "not to the conf.py location. Use a starting '/', like '/needs.json', to make the path "
                             "relative to conf.py.",
-                            None,
+                            "deprecated",
                             location=(self.env.docname, self.lineno),
                         )
             else:
@@ -181,7 +181,7 @@ class NeedimportDirective(SphinxDirective):
                     log_warning(
                         logger,
                         f"needimport: Filter {filter_string} not valid. Error: {e}. {self.docname}{self.lineno}",
-                        None,
+                        "needimport",
                         location=(self.env.docname, self.lineno),
                     )
 
@@ -252,14 +252,23 @@ class NeedimportDirective(SphinxDirective):
             )  # type: ignore[misc]
 
             # Replace id, to get unique ids
-            need_params["id"] = id_prefix + need_params["id"]
+            need_id = need_params["id"] = id_prefix + need_params["id"]
 
             # override location
             need_params["docname"] = self.docname
             need_params["lineno"] = self.lineno
 
-            nodes = add_need(self.env.app, self.state, **need_params)  # type: ignore[call-arg]
-            need_nodes.extend(nodes)
+            try:
+                nodes = add_need(self.env.app, self.state, **need_params)  # type: ignore[call-arg]
+            except InvalidNeedException as err:
+                log_warning(
+                    logger,
+                    f"Need {need_id!r} could not be imported: {err.message}",
+                    "import_need",
+                    location=self.get_location(),
+                )
+            else:
+                need_nodes.extend(nodes)
 
         if unknown_keys:
             log_warning(
diff --git a/sphinx_needs/directives/needreport.py b/sphinx_needs/directives/needreport.py
index 5b8b4419..270c5dfa 100644
--- a/sphinx_needs/directives/needreport.py
+++ b/sphinx_needs/directives/needreport.py
@@ -34,7 +34,7 @@ class NeedReportDirective(SphinxDirective):
             log_warning(
                 LOGGER,
                 "No options specified to generate need report",
-                "report",
+                "needreport",
                 location=self.get_location(),
             )
             return []
@@ -73,7 +73,7 @@ class NeedReportDirective(SphinxDirective):
             log_warning(
                 LOGGER,
                 f"Could not load needs report template file {need_report_template_path}",
-                "report",
+                "needreport",
                 location=self.get_location(),
             )
             return []
diff --git a/sphinx_needs/directives/needsequence.py b/sphinx_needs/directives/needsequence.py
index 4e239029..a1c16d6c 100644
--- a/sphinx_needs/directives/needsequence.py
+++ b/sphinx_needs/directives/needsequence.py
@@ -116,8 +116,8 @@ def process_needsequence(
                         flow=current_needsequence["target_id"],
                         link_types=",".join(link_type_names),
                     ),
-                    None,
-                    None,
+                    "needsequence",
+                    location=node,
                 )
 
         content = []
diff --git a/sphinx_needs/directives/needservice.py b/sphinx_needs/directives/needservice.py
index 47f6360b..1d4b0ee3 100644
--- a/sphinx_needs/directives/needservice.py
+++ b/sphinx_needs/directives/needservice.py
@@ -9,11 +9,11 @@ from docutils.statemachine import StringList
 from sphinx.util.docutils import SphinxDirective
 from sphinx_data_viewer.api import get_data_viewer_node
 
-from sphinx_needs.api import add_need
+from sphinx_needs.api import InvalidNeedException, add_need
 from sphinx_needs.config import NeedsSphinxConfig
 from sphinx_needs.data import SphinxNeedsData
 from sphinx_needs.directives.need import NeedDirective
-from sphinx_needs.logging import get_logger
+from sphinx_needs.logging import get_logger, log_warning
 from sphinx_needs.utils import add_doc
 
 
@@ -126,15 +126,23 @@ class NeedserviceDirective(SphinxDirective):
                 datum.update(options)
 
                 # ToDo: Tags and Status are not set (but exist in data)
-                section += add_need(
-                    self.env.app,
-                    self.state,
-                    docname,
-                    self.lineno,
-                    need_type,
-                    need_title,
-                    **datum,
-                )
+                try:
+                    section += add_need(
+                        self.env.app,
+                        self.state,
+                        docname,
+                        self.lineno,
+                        need_type,
+                        need_title,
+                        **datum,
+                    )
+                except InvalidNeedException as err:
+                    log_warning(
+                        self.log,
+                        f"Service need could not be created: {err.message}",
+                        "load_service_need",
+                        location=self.get_location(),
+                    )
         else:
             try:
                 service_debug_data = service.debug(self.options)
diff --git a/sphinx_needs/external_needs.py b/sphinx_needs/external_needs.py
index 0283b65f..f56d301b 100644
--- a/sphinx_needs/external_needs.py
+++ b/sphinx_needs/external_needs.py
@@ -10,7 +10,7 @@ from requests_file import FileAdapter
 from sphinx.application import Sphinx
 from sphinx.environment import BuildEnvironment
 
-from sphinx_needs.api import add_external_need, del_need
+from sphinx_needs.api import InvalidNeedException, add_external_need, del_need
 from sphinx_needs.config import NEEDS_CONFIG, NeedsSphinxConfig
 from sphinx_needs.data import NeedsCoreFields, SphinxNeedsData
 from sphinx_needs.logging import get_logger, log_warning
@@ -29,7 +29,9 @@ def get_target_template(target_url: str) -> Template:
     return mem_template
 
 
-def load_external_needs(app: Sphinx, env: BuildEnvironment, docname: str) -> None:
+def load_external_needs(
+    app: Sphinx, env: BuildEnvironment, _docnames: list[str]
+) -> None:
     """Load needs from configured external sources."""
     needs_config = NeedsSphinxConfig(app.config)
     for source in needs_config.external_needs:
@@ -175,30 +177,32 @@ def load_external_needs(app: Sphinx, env: BuildEnvironment, docname: str) -> Non
 
             need = SphinxNeedsData(env).get_needs_mutable().get(ext_need_id)
 
-            if need is not None:
-                # check need_params for more detail
-                if need["is_external"] and source["base_url"] in need["external_url"]:
-                    # delete the already existing external need from api need
-                    del_need(app, ext_need_id)
-                else:
-                    log_warning(
-                        log,
-                        f'During external needs handling, an identical ID was detected: {ext_need_id} '
-                        f'from needs_external_needs url: {source["base_url"]}',
-                        "duplicate_id",
-                        location=docname if docname else None,
-                    )
-                    return None
+            if (
+                need is not None
+                and need["is_external"]
+                and source["base_url"] in need["external_url"]
+            ):
+                # delete the already existing external need from api need
+                del_need(app, ext_need_id)
 
-            add_external_need(app, **need_params)
+            try:
+                add_external_need(app, **need_params)
+            except InvalidNeedException as err:
+                location = source.get("json_url", "") or source.get("json_path", "")
+                log_warning(
+                    log,
+                    f"External need {ext_need_id!r} in {location!r} could not be added: {err.message}",
+                    "load_external_need",
+                    location=None,
+                )
 
         if unknown_keys:
             location = source.get("json_url", "") or source.get("json_path", "")
             log_warning(
                 log,
-                f"Unknown keys in external need source: {sorted(unknown_keys)!r}",
+                f"Unknown keys in external need source {location!r}: {sorted(unknown_keys)!r}",
                 "unknown_external_keys",
-                location=location,
+                location=None,
             )
 
 
diff --git a/sphinx_needs/filter_common.py b/sphinx_needs/filter_common.py
index cf02be76..e08decdf 100644
--- a/sphinx_needs/filter_common.py
+++ b/sphinx_needs/filter_common.py
@@ -211,7 +211,7 @@ def process_filters(
             )
         else:
             log_warning(
-                log, "Something went wrong running filter", None, location=location
+                log, "Something went wrong running filter", "filter", location=location
             )
             return []
 
@@ -235,7 +235,7 @@ def process_filters(
             log_warning(
                 log,
                 f"Sorting parameter {sort_key} not valid: Error: {e}",
-                None,
+                "filter",
                 location=location,
             )
             return []
diff --git a/sphinx_needs/functions/common.py b/sphinx_needs/functions/common.py
index 6209b967..5db92bba 100644
--- a/sphinx_needs/functions/common.py
+++ b/sphinx_needs/functions/common.py
@@ -287,7 +287,7 @@ def check_linked_values(
                 log_warning(
                     logger,
                     f"CheckLinkedValues: Filter {filter_string} not valid: Error: {e}",
-                    None,
+                    "filter",
                     None,
                 )
 
@@ -392,7 +392,10 @@ def calc_sum(
                 pass
             except NeedsInvalidFilter as ex:
                 log_warning(
-                    logger, f"Given filter is not valid. Error: {ex}", None, None
+                    logger,
+                    f"Given filter is not valid. Error: {ex}",
+                    "filter",
+                    None,
                 )
 
         with contextlib.suppress(ValueError):
diff --git a/sphinx_needs/functions/functions.py b/sphinx_needs/functions/functions.py
index cd7ce4c9..dc7fce80 100644
--- a/sphinx_needs/functions/functions.py
+++ b/sphinx_needs/functions/functions.py
@@ -201,7 +201,7 @@ def find_and_replace_node_content(
             func_string = func_string.replace("’", "'")  # noqa: RUF001
 
             msg = f"The [[{func_string}]] syntax in need content is deprecated. Replace with :ndf:`{func_string}` instead."
-            log_warning(logger, msg, "deprecation", location=node)
+            log_warning(logger, msg, "deprecated", location=node)
 
             func_return = execute_func(
                 env.app, need, SphinxNeedsData(env).get_needs_view(), func_string, node
diff --git a/sphinx_needs/logging.py b/sphinx_needs/logging.py
index 22ec314c..5248c99c 100644
--- a/sphinx_needs/logging.py
+++ b/sphinx_needs/logging.py
@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+from typing import Literal
+
 from docutils.nodes import Node
 from sphinx import version_info
 from sphinx.util import logging
@@ -10,10 +12,86 @@ def get_logger(name: str) -> SphinxLoggerAdapter:
     return logging.getLogger(name)
 
 
+WarningSubTypes = Literal[
+    "config",
+    "constraint",
+    "create_need",
+    "delete_need",
+    "deprecated",
+    "diagram_scale",
+    "duplicate_id",
+    "duplicate_part_id",
+    "dynamic_function",
+    "external_link_outgoing",
+    "needextend",
+    "needextract",
+    "needflow",
+    "needgantt",
+    "needimport",
+    "needreport",
+    "needsequence",
+    "filter",
+    "filter_func",
+    "github",
+    "import_need",
+    "json_load",
+    "layout",
+    "link_outgoing",
+    "link_ref",
+    "link_text",
+    "load_external_need",
+    "load_service_need",
+    "mpl",
+    "title",
+    "uml",
+    "unknown_external_keys",
+    "unknown_import_keys",
+    "variant",
+    "warnings",
+]
+
+WarningSubTypeDescription: dict[WarningSubTypes, str] = {
+    "config": "Invalid configuration",
+    "constraint": "Constraint violation",
+    "create_need": "Creation of a need from directive failed",
+    "delete_need": "Deletion of a need failed",
+    "deprecated": "Deprecated feature",
+    "diagram_scale": "Failed to process diagram scale option",
+    "duplicate_id": "Duplicate need ID found when merging needs from parallel processes",
+    "duplicate_part_id": "Duplicate part ID found when parsing need content",
+    "dynamic_function": "Failed to load/execute dynamic function",
+    "needextend": "Error processing needextend directive",
+    "needextract": "Error processing needextract directive",
+    "needflow": "Error processing needflow directive",
+    "needgantt": "Error processing needgantt directive",
+    "needimport": "Error processing needimport directive",
+    "needreport": "Error processing needreport directive",
+    "needsequence": "Error processing needsequence directive",
+    "filter": "Error processing needs filter",
+    "filter_func": "Error loading needs filter function",
+    "github": "Error in processing GitHub service directive",
+    "import_need": "Failed to import a need",
+    "layout": "Error occurred during layout rendering of a need",
+    "link_outgoing": "Unknown outgoing link in standard need",
+    "external_link_outgoing": "Unknown outgoing link in external need",
+    "link_ref": "Need could not be referenced",
+    "link_text": "Reference text could not be generated",
+    "load_external_need": "Failed to load an external need",
+    "load_service_need": "Failed to load a service need",
+    "mpl": "Matplotlib required but not installed",
+    "title": "Error creating need title",
+    "uml": "Error in processing of UML diagram",
+    "unknown_external_keys": "Unknown keys found in external need data",
+    "unknown_import_keys": "Unknown keys found in imported need data",
+    "variant": "Error processing variant in need field",
+    "warnings": "Need warning check failed for one or more needs",
+}
+
+
 def log_warning(
     logger: SphinxLoggerAdapter,
     message: str,
-    subtype: str | None,
+    subtype: WarningSubTypes,
     /,
     location: str | tuple[str | None, int | None] | Node | None,
     *,
diff --git a/sphinx_needs/need_constraints.py b/sphinx_needs/need_constraints.py
index ce3c7651..c11d207f 100644
--- a/sphinx_needs/need_constraints.py
+++ b/sphinx_needs/need_constraints.py
@@ -2,7 +2,7 @@ from __future__ import annotations
 
 import jinja2
 
-from sphinx_needs.api.exceptions import NeedsConstraintFailed, NeedsConstraintNotAllowed
+from sphinx_needs.api.exceptions import NeedsConstraintFailed
 from sphinx_needs.config import NeedsSphinxConfig
 from sphinx_needs.data import NeedsMutable
 from sphinx_needs.filter_common import filter_single_need
@@ -35,9 +35,7 @@ def process_constraints(needs: NeedsMutable, config: NeedsSphinxConfig) -> None:
                 executable_constraints = config_constraints[constraint]
             except KeyError:
                 # Note, this is already checked for in add_need
-                raise NeedsConstraintNotAllowed(
-                    f"Constraint {constraint} of need id {need_id} is not allowed by config value 'needs_constraints'."
-                )
+                continue
 
             # name is check_0, check_1, ...
             for name, cmd in executable_constraints.items():
diff --git a/sphinx_needs/needs.py b/sphinx_needs/needs.py
index f647079f..54e8fc43 100644
--- a/sphinx_needs/needs.py
+++ b/sphinx_needs/needs.py
@@ -499,7 +499,7 @@ def visitor_dummy(*_args: Any, **_kwargs: Any) -> None:
     pass
 
 
-def prepare_env(app: Sphinx, env: BuildEnvironment, _docname: str) -> None:
+def prepare_env(app: Sphinx, env: BuildEnvironment, _docnames: list[str]) -> None:
     """
     Prepares the sphinx environment to store sphinx-needs internal data.
     """
diff --git a/sphinx_needs/needsfile.py b/sphinx_needs/needsfile.py
index c8889ef3..62ac8c0a 100644
--- a/sphinx_needs/needsfile.py
+++ b/sphinx_needs/needsfile.py
@@ -177,28 +177,33 @@ class NeedsList:
         if version in self.needs_list["versions"]:
             del self.needs_list["versions"][version]
 
-    def write_json(self, needs_file: str = "needs.json", needs_path: str = "") -> None:
-        # We need to rewrite some data, because this kind of data gets overwritten during needs.json import.
+    def _finalise(self) -> None:
+        # We need to rewrite some data, because this kind of data gets overwritten during needs.json import
         if not self.needs_config.reproducible_json:
             self.needs_list["created"] = datetime.now().isoformat()
         else:
             self.needs_list.pop("created", None)
         self.needs_list["current_version"] = self.current_version
         self.needs_list["project"] = self.project
-        if needs_path:
-            needs_dir = needs_path
-        else:
-            needs_dir = self.outdir
 
+    def write_json(self, needs_file: str = "needs.json", needs_path: str = "") -> None:
+        self._finalise()
+        needs_dir = needs_path if needs_path else self.outdir
         with open(os.path.join(needs_dir, needs_file), "w") as f:
             json.dump(self.needs_list, f, sort_keys=True)
 
+    def dump_json(self) -> str:
+        self._finalise()
+        return json.dumps(self.needs_list, sort_keys=True)
+
     def load_json(self, file: str) -> None:
         if not os.path.isabs(file):
             file = os.path.join(self.confdir, file)
 
         if not os.path.exists(file):
-            log_warning(self.log, f"Could not load needs json file {file}", None, None)
+            log_warning(
+                self.log, f"Could not load needs json file {file}", "json_load", None
+            )
         else:
             errors = check_needs_file(file)
             # We only care for schema errors here, all other possible errors
@@ -213,7 +218,10 @@ class NeedsList:
                     needs_list = json.load(needs_file)
                 except json.JSONDecodeError:
                     log_warning(
-                        self.log, f"Could not decode json file {file}", None, None
+                        self.log,
+                        f"Could not decode json file {file}",
+                        "json_load",
+                        None,
                     )
                 else:
                     self.needs_list = needs_list
diff --git a/sphinx_needs/roles/need_func.py b/sphinx_needs/roles/need_func.py
index d5d09ae1..d9a3850f 100644
--- a/sphinx_needs/roles/need_func.py
+++ b/sphinx_needs/roles/need_func.py
@@ -45,7 +45,7 @@ class NeedFuncRole(SphinxRole):
                 msg += f"Replace with :ndf:`{func_call}` instead."
             else:
                 msg += "Replace with ndf role instead."
-            log_warning(LOGGER, msg, "deprecation", location=node)
+            log_warning(LOGGER, msg, "deprecated", location=node)
         return [node], []
 
 
diff --git a/sphinx_needs/roles/need_incoming.py b/sphinx_needs/roles/need_incoming.py
index 98453724..8d72557b 100644
--- a/sphinx_needs/roles/need_incoming.py
+++ b/sphinx_needs/roles/need_incoming.py
@@ -95,7 +95,7 @@ def process_need_incoming(
                 log_warning(
                     logger,
                     f"need {node_need_backref['reftarget']} not found",
-                    None,
+                    "link_ref",
                     location=node_need_backref,
                 )
 
diff --git a/sphinx_needs/roles/need_part.py b/sphinx_needs/roles/need_part.py
index c4b9e660..6db75518 100644
--- a/sphinx_needs/roles/need_part.py
+++ b/sphinx_needs/roles/need_part.py
@@ -86,8 +86,8 @@ def update_need_with_parts(
                 "part_need id {} in need {} is already taken. need_part may get overridden.".format(
                     inline_id, need["id"]
                 ),
-                None,
-                None,
+                "duplicate_part_id",
+                part_node,
             )
 
         need["parts"][inline_id] = {
diff --git a/sphinx_needs/roles/need_ref.py b/sphinx_needs/roles/need_ref.py
index 00286793..bd0453a0 100644
--- a/sphinx_needs/roles/need_ref.py
+++ b/sphinx_needs/roles/need_ref.py
@@ -106,6 +106,7 @@ def process_need_ref(
             if str(need_id_full) == str(ref_name):
                 ref_name = None
 
+            link_text = ""
             if ref_name and prefix in ref_name and postfix in ref_name:
                 # if ref_name is set and has prefix to process, we will do so.
                 ref_name = ref_name.replace(prefix, "{").replace(postfix, "}")
@@ -115,7 +116,7 @@ def process_need_ref(
                     log_warning(
                         log,
                         f"option placeholder {e} for need {node_need_ref['reftarget']} not found",
-                        None,
+                        "link_text",
                         location=node_need_ref,
                     )
             else:
@@ -125,8 +126,12 @@ def process_need_ref(
                 try:
                     link_text = needs_config.role_need_template.format(**dict_need)
                 except KeyError as e:
-                    link_text = f'"the config parameter needs_role_need_template uses not supported placeholders: {e} "'
-                    log_warning(log, link_text, None, None)
+                    log_warning(
+                        log,
+                        f"the config parameter needs_role_need_template uses unsupported placeholders: {e} ",
+                        "link_text",
+                        location=node_need_ref,
+                    )
 
             node_need_ref[0].children[0] = nodes.Text(link_text)  # type: ignore[index]
 
diff --git a/sphinx_needs/utils.py b/sphinx_needs/utils.py
index ad90b314..06c44c13 100644
--- a/sphinx_needs/utils.py
+++ b/sphinx_needs/utils.py
@@ -496,7 +496,7 @@ def match_string_link(
             logger,
             f'Problems dealing with string to link transformation for value "{data}" of '
             f'option "{need_key}". Error: {e}',
-            None,
+            "layout",
             None,
         )
     else:
@@ -649,24 +649,7 @@ def add_doc(env: BuildEnvironment, docname: str, category: str | None = None) ->
 
 def split_link_types(link_types: str, location: Any) -> list[str]:
     """Split link_types string into list of link_types."""
-
-    def _is_valid(link_type: str) -> bool:
-        if len(link_type) == 0 or link_type.isspace():
-            log_warning(
-                logger,
-                "Scruffy link_type definition found. Defined link_type contains spaces only.",
-                None,
-                location=location,
-            )
-            return False
-        return True
-
-    return list(
-        filter(
-            _is_valid,
-            (x.strip() for x in re.split(";|,", link_types)),
-        )
-    )
+    return [x.strip() for x in re.split(";|,", link_types) if x.strip()]
 
 
 def get_scale(options: dict[str, Any], location: Any) -> str:
@@ -676,7 +659,7 @@ def get_scale(options: dict[str, Any], location: Any) -> str:
         log_warning(
             logger,
             f'scale value must be a number. "{scale}" found',
-            None,
+            "diagram_scale",
             location=location,
         )
         return "100"
@@ -684,7 +667,7 @@ def get_scale(options: dict[str, Any], location: Any) -> str:
         log_warning(
             logger,
             f'scale value must be between 1 and 300. "{scale}" found',
-            None,
+            "diagram_scale",
             location=location,
         )
         return "100"


Test patch:
diff --git a/tests/doc_test/need_constraints_failed/conf.py b/tests/doc_test/need_constraints_failed/conf.py
index fcbcb277..64873d96 100644
--- a/tests/doc_test/need_constraints_failed/conf.py
+++ b/tests/doc_test/need_constraints_failed/conf.py
@@ -1,91 +1,1 @@
 extensions = ["sphinx_needs"]
-
-needs_table_style = "TABLE"
-
-needs_types = [
-    {
-        "directive": "story",
-        "title": "User Story",
-        "prefix": "US_",
-        "color": "#BFD8D2",
-        "style": "node",
-    },
-    {
-        "directive": "spec",
-        "title": "Specification",
-        "prefix": "SP_",
-        "color": "#FEDCD2",
-        "style": "node",
-    },
-    {
-        "directive": "impl",
-        "title": "Implementation",
-        "prefix": "IM_",
-        "color": "#DF744A",
-        "style": "node",
-    },
-    {
-        "directive": "test",
-        "title": "Test Case",
-        "prefix": "TC_",
-        "color": "#DCB239",
-        "style": "node",
-    },
-]
-
-needs_external_needs = [
-    {
-        "base_url": "http://my_company.com/docs/v1/",
-        "json_path": "needs_test_small.json",
-        "id_prefix": "ext_",
-    }
-]
-
-
-def my_custom_warning_check(need, log):
-    if need["status"] == "open":
-        log.info(f"{need['id']} status must not be 'open'.")
-        return True
-    return False
-
-
-needs_warnings = {
-    "invalid_status": "status not in ['open', 'closed', 'done', 'example_2', 'example_3']",
-    "type_match": my_custom_warning_check,
-}
-
-
-def custom_warning_func(need, log):
-    return need["status"] == "example_3"
-
-
-def setup(app):
-    from sphinx_needs.api.configuration import add_warning
-
-    add_warning(app, "api_warning_filter", filter_string="status == 'example_2'")
-    add_warning(app, "api_warning_func", custom_warning_func)
-    add_warning(
-        app,
-        "invalid_status",
-        "status not in ['open', 'closed', 'done', 'example_2', 'example_3']",
-    )
-
-
-# Needs option to set True or False to raise sphinx-warning for each not passed warning check
-# default is False
-needs_warnings_always_warn = True
-
-needs_extra_options = []
-
-needs_constraints = {
-    "security": {"check_0": "'security' in tags", "severity": "CRITICAL"},
-    "team": {"check_0": "'team_requirement' in links", "severity": "MEDIUM"},
-    "critical": {"check_0": "'critical' in tags", "severity": "CRITICAL"},
-}
-
-needs_constraint_failed_options = {
-    "CRITICAL": {"on_fail": ["warn"], "style": ["red_bar"], "force_style": True},
-    "HIGH": {"on_fail": ["warn"], "style": ["orange_bar"], "force_style": True},
-    "MEDIUM": {"on_fail": ["warn"], "style": ["yellow_bar"], "force_style": False},
-    "LOW": {"on_fail": [], "style": ["yellow_bar"], "force_style": False},
-}
diff --git a/tests/doc_test/need_constraints_failed/index.rst b/tests/doc_test/need_constraints_failed/index.rst
index 8b7fd1a5..61056969 100644
--- a/tests/doc_test/need_constraints_failed/index.rst
+++ b/tests/doc_test/need_constraints_failed/index.rst
@@ -1,7 +1,7 @@
 TEST DOCUMENT NEEDS CONSTRAINTS
-============================
+===============================
 
 .. spec:: FAIL_03
     :constraints: non_existing
 
-    Example of a failed constraint with medium severity. Note the style from :ref:`needs_constraint_failed_options`
\ No newline at end of file
+    Content...
\ No newline at end of file
diff --git a/tests/doc_test/need_constraints_failed/needs_test_small.json b/tests/doc_test/need_constraints_failed/needs_test_small.json
deleted file mode 100644
index 1ee76829..00000000
--- a/tests/doc_test/need_constraints_failed/needs_test_small.json
+++ /dev/null
@@ -1,52 +0,0 @@
-{
-    "created": "2017-08-25T21:17:17.640061",
-    "current_version": "0.1.33",
-    "project": "Sphinx-Needs",
-    "versions": {
-        "0.1.33": {
-            "created": "2017-08-25T21:17:17.640058",
-            "needs": {
-                "TEST_01": {
-                    "description": "TEST_01",
-                    "id": "TEST_01",
-                    "links": [],
-                    "status": null,
-                    "tags": [],
-                    "title": "TEST_01 DESCRIPTION",
-                    "type": "impl",
-                    "type_name": "Implementation"
-                },
-                "TEST_02": {
-                    "description": "TEST_02",
-                    "id": "TEST_02",
-                    "links": [],
-                    "status": "open",
-                    "tags": ["test_02", "test"],
-                    "title": "TEST_02 DESCRIPTION",
-                    "type": "test",
-                    "type_name": "Test Case"
-                },
-                "TEST_03": {
-                    "description": "AAA",
-                    "id": "TEST_03",
-                    "links": [],
-                    "status": "closed",
-                    "tags": [],
-                    "title": "AAA",
-                    "type": "spec",
-                    "type_name": "Specification"
-                },
-                "UI improvements": {
-                    "description": "develop a darkmode",
-                    "id": "TEST_04",
-                    "links": ["TEST_02"],
-                    "status": "dev",
-                    "tags": ["tag1", "tag2", "asd"],
-                    "title": "",
-                    "type": "spec",
-                    "type_name": "Specification"
-                }
-            }
-        }
-    }
-}
\ No newline at end of file
diff --git a/tests/test_basic_doc.py b/tests/test_basic_doc.py
index 81d5a81e..450598c6 100644
--- a/tests/test_basic_doc.py
+++ b/tests/test_basic_doc.py
@@ -11,8 +11,6 @@ from sphinx import version_info
 from sphinx.testing.util import SphinxTestApp
 from syrupy.filters import props
 
-from sphinx_needs.api.need import NeedsNoIdException
-
 
 @pytest.mark.parametrize(
     "test_app",
@@ -152,25 +150,6 @@ def test_build_needs(test_app: SphinxTestApp, snapshot):
     assert needs_data == snapshot(exclude=props("created", "project", "creator"))
 
 
-# Test with needs_id_required=True and missing ids in docs.
-@pytest.mark.parametrize(
-    "test_app",
-    [
-        {
-            "buildername": "html",
-            "srcdir": "doc_test/doc_basic",
-            "confoverrides": {"needs_id_required": True},
-            "no_plantuml": True,
-        }
-    ],
-    indirect=True,
-)
-def test_id_required_build_html(test_app: SphinxTestApp):
-    with pytest.raises(NeedsNoIdException):
-        app = test_app
-        app.build()
-
-
 def test_sphinx_api_build(tmp_path: Path, make_app: type[SphinxTestApp]):
     """
     Tests a build via the Sphinx Build API.
diff --git a/tests/test_broken_docs.py b/tests/test_broken_docs.py
index 0f79e211..7aae0350 100644
--- a/tests/test_broken_docs.py
+++ b/tests/test_broken_docs.py
@@ -5,8 +5,6 @@ import pytest
 from sphinx.testing.util import SphinxTestApp
 from sphinx.util.console import strip_colors
 
-from sphinx_needs.api.need import NeedsStatusNotAllowed, NeedsTagNotAllowed
-
 
 def get_warnings(app: SphinxTestApp):
     return (
@@ -16,6 +14,25 @@ def get_warnings(app: SphinxTestApp):
     )
 
 
+@pytest.mark.parametrize(
+    "test_app",
+    [
+        {
+            "buildername": "html",
+            "srcdir": "doc_test/doc_basic",
+            "confoverrides": {"needs_id_required": True},
+            "no_plantuml": True,
+        }
+    ],
+    indirect=True,
+)
+def test_id_required_build_html(test_app: SphinxTestApp):
+    test_app.build()
+    assert get_warnings(test_app) == [
+        "<srcdir>/index.rst:8: WARNING: Need could not be created: No ID defined, but 'needs_id_required' is set to True. [needs.create_need]"
+    ]
+
+
 @pytest.mark.parametrize(
     "test_app",
     [{"buildername": "html", "srcdir": "doc_test/broken_doc", "no_plantuml": True}],
@@ -24,7 +41,7 @@ def get_warnings(app: SphinxTestApp):
 def test_duplicate_id(test_app: SphinxTestApp):
     test_app.build()
     assert get_warnings(test_app) == [
-        "<srcdir>/index.rst:11: WARNING: A need with ID SP_TOO_001 already exists, title: 'Command line interface'. [needs.duplicate_id]"
+        "<srcdir>/index.rst:11: WARNING: Need could not be created: A need with ID 'SP_TOO_001' already exists. [needs.create_need]"
     ]
     html = (test_app.outdir / "index.html").read_text()
     assert "<h1>BROKEN DOCUMENT" in html
@@ -48,12 +65,20 @@ def test_broken_links(test_app: SphinxTestApp):
 
 @pytest.mark.parametrize(
     "test_app",
-    [{"buildername": "html", "srcdir": "doc_test/broken_statuses"}],
+    [
+        {
+            "buildername": "html",
+            "srcdir": "doc_test/broken_statuses",
+            "no_plantuml": True,
+        }
+    ],
     indirect=True,
 )
 def test_broken_statuses(test_app: SphinxTestApp):
-    with pytest.raises(NeedsStatusNotAllowed):
-        test_app.build()
+    test_app.build()
+    assert get_warnings(test_app) == [
+        "<srcdir>/index.rst:11: WARNING: Need could not be created: Status 'NOT_ALLOWED' not in 'needs_statuses'. [needs.create_need]"
+    ]
 
 
 @pytest.mark.parametrize(
@@ -84,9 +109,11 @@ def test_broken_syntax(test_app: SphinxTestApp):
 
 @pytest.mark.parametrize(
     "test_app",
-    [{"buildername": "html", "srcdir": "doc_test/broken_tags"}],
+    [{"buildername": "html", "srcdir": "doc_test/broken_tags", "no_plantuml": True}],
     indirect=True,
 )
 def test_broken_tags(test_app: SphinxTestApp):
-    with pytest.raises(NeedsTagNotAllowed):
-        test_app.build()
+    test_app.build()
+    assert get_warnings(test_app) == [
+        "<srcdir>/index.rst:17: WARNING: Need could not be created: Tags {'BROKEN'} not in 'needs_tags'. [needs.create_need]"
+    ]
diff --git a/tests/test_dynamic_functions.py b/tests/test_dynamic_functions.py
index caef56c7..eb0c8ea4 100644
--- a/tests/test_dynamic_functions.py
+++ b/tests/test_dynamic_functions.py
@@ -26,13 +26,13 @@ def test_doc_dynamic_functions(test_app):
         app._warning.getvalue().replace(str(app.srcdir) + os.sep, "srcdir/")
     ).splitlines()
     assert warnings == [
-        'srcdir/index.rst:11: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:40: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:44: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:9: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:27: WARNING: The [[copy("tags")]] syntax in need content is deprecated. Replace with :ndf:`copy("tags")` instead. [needs.deprecation]',
-        "srcdir/index.rst:33: WARNING: The [[copy('id')]] syntax in need content is deprecated. Replace with :ndf:`copy('id')` instead. [needs.deprecation]",
-        "srcdir/index.rst:38: WARNING: The [[copy('id')]] syntax in need content is deprecated. Replace with :ndf:`copy('id')` instead. [needs.deprecation]",
+        'srcdir/index.rst:11: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:40: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:44: WARNING: The `need_func` role is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:9: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:27: WARNING: The [[copy("tags")]] syntax in need content is deprecated. Replace with :ndf:`copy("tags")` instead. [needs.deprecated]',
+        "srcdir/index.rst:33: WARNING: The [[copy('id')]] syntax in need content is deprecated. Replace with :ndf:`copy('id')` instead. [needs.deprecated]",
+        "srcdir/index.rst:38: WARNING: The [[copy('id')]] syntax in need content is deprecated. Replace with :ndf:`copy('id')` instead. [needs.deprecated]",
         "srcdir/index.rst:44: WARNING: Error while executing function 'copy': Need not found [needs.dynamic_function]",
         "srcdir/index.rst:44: WARNING: Error while executing function 'copy': Need not found [needs.dynamic_function]",
     ]
@@ -128,12 +128,12 @@ def test_doc_df_user_functions(test_app):
     # print(warnings)
     expected = [
         "srcdir/index.rst:10: WARNING: Return value of function 'bad_function' is of type <class 'object'>. Allowed are str, int, float, list [needs.dynamic_function]",
-        "srcdir/index.rst:8: WARNING: The [[my_own_function()]] syntax in need content is deprecated. Replace with :ndf:`my_own_function()` instead. [needs.deprecation]",
-        "srcdir/index.rst:14: WARNING: The [[bad_function()]] syntax in need content is deprecated. Replace with :ndf:`bad_function()` instead. [needs.deprecation]",
+        "srcdir/index.rst:8: WARNING: The [[my_own_function()]] syntax in need content is deprecated. Replace with :ndf:`my_own_function()` instead. [needs.deprecated]",
+        "srcdir/index.rst:14: WARNING: The [[bad_function()]] syntax in need content is deprecated. Replace with :ndf:`bad_function()` instead. [needs.deprecated]",
         "srcdir/index.rst:14: WARNING: Return value of function 'bad_function' is of type <class 'object'>. Allowed are str, int, float, list [needs.dynamic_function]",
-        "srcdir/index.rst:16: WARNING: The [[invalid]] syntax in need content is deprecated. Replace with :ndf:`invalid` instead. [needs.deprecation]",
+        "srcdir/index.rst:16: WARNING: The [[invalid]] syntax in need content is deprecated. Replace with :ndf:`invalid` instead. [needs.deprecated]",
         "srcdir/index.rst:16: WARNING: Function string 'invalid' could not be parsed: Given dynamic function string is not a valid python call. Got: invalid [needs.dynamic_function]",
-        "srcdir/index.rst:18: WARNING: The [[unknown()]] syntax in need content is deprecated. Replace with :ndf:`unknown()` instead. [needs.deprecation]",
+        "srcdir/index.rst:18: WARNING: The [[unknown()]] syntax in need content is deprecated. Replace with :ndf:`unknown()` instead. [needs.deprecated]",
         "srcdir/index.rst:18: WARNING: Unknown function 'unknown' [needs.dynamic_function]",
     ]
     if version_info >= (7, 3):
diff --git a/tests/test_external.py b/tests/test_external.py
index 37e40ed9..c97289ba 100644
--- a/tests/test_external.py
+++ b/tests/test_external.py
@@ -21,8 +21,7 @@ def test_external_html(test_app: SphinxTestApp):
     app = test_app
     app.build()
     assert strip_colors(app._warning.getvalue()).strip() == (
-        "WARNING: Couldn't create need EXT_TEST_03. "
-        "Reason: The need-type (i.e. `ask`) is not set in the project's 'need_types' configuration in conf.py. [needs.add]"
+        "WARNING: External need 'EXT_TEST_03' in 'needs_test_small.json' could not be added: Unknown need type 'ask'. [needs.load_external_need]"
     )
     html = Path(app.outdir, "index.html").read_text()
     assert (
diff --git a/tests/test_filter.py b/tests/test_filter.py
index 71935452..6ae44fe6 100644
--- a/tests/test_filter.py
+++ b/tests/test_filter.py
@@ -26,7 +26,7 @@ def test_filter_build_html(test_app):
         "<srcdir>/index.rst:51: WARNING: Filter 'xxx' not valid. Error: name 'xxx' is not defined. [needs.filter]",
         "<srcdir>/index.rst:54: WARNING: Filter '1' not valid. Error: Filter did not evaluate to a boolean, instead <class 'int'>: 1. [needs.filter]",
         "<srcdir>/index.rst:57: WARNING: Filter 'yyy' not valid. Error: name 'yyy' is not defined. [needs.filter]",
-        "<srcdir>/index.rst:60: WARNING: Sorting parameter yyy not valid: Error: 'yyy' [needs]",
+        "<srcdir>/index.rst:60: WARNING: Sorting parameter yyy not valid: Error: 'yyy' [needs.filter]",
         "<srcdir>/index.rst:63: WARNING: Filter 'zzz' not valid. Error: name 'zzz' is not defined. [needs.filter]",
     ]
 
diff --git a/tests/test_need_constraints.py b/tests/test_need_constraints.py
index febf8760..ce2a60b9 100644
--- a/tests/test_need_constraints.py
+++ b/tests/test_need_constraints.py
@@ -1,12 +1,12 @@
 import json
+import os
 import subprocess
 from pathlib import Path
 
 import pytest
+from sphinx.util.console import strip_colors
 from syrupy.filters import props
 
-from sphinx_needs.api.exceptions import NeedsConstraintNotAllowed
-
 
 @pytest.mark.parametrize(
     "test_app",
@@ -55,12 +55,22 @@ def test_need_constraints(test_app, snapshot):
 
 @pytest.mark.parametrize(
     "test_app",
-    [{"buildername": "html", "srcdir": "doc_test/need_constraints_failed"}],
+    [
+        {
+            "buildername": "html",
+            "srcdir": "doc_test/need_constraints_failed",
+            "no_plantuml": True,
+        }
+    ],
     indirect=True,
 )
 def test_need_constraints_config(test_app):
-    app = test_app
-
-    # check that correct exception is raised
-    with pytest.raises(NeedsConstraintNotAllowed):
-        app.build()
+    test_app.build()
+    warnings = (
+        strip_colors(test_app._warning.getvalue())
+        .replace(str(test_app.srcdir) + os.path.sep, "<srcdir>/")
+        .splitlines()
+    )
+    assert warnings == [
+        "<srcdir>/index.rst:4: WARNING: Need could not be created: Constraints {'non_existing'} not in 'needs_constraints'. [needs.create_need]"
+    ]
diff --git a/tests/test_needextend.py b/tests/test_needextend.py
index 630f40d3..80f1b6ed 100644
--- a/tests/test_needextend.py
+++ b/tests/test_needextend.py
@@ -62,7 +62,7 @@ def test_doc_needextend_unknown_id(test_app: Sphinx):
 
     warnings = strip_colors(app._warning.getvalue()).splitlines()
     assert warnings == [
-        f"{Path(str(app.srcdir)) / 'index.rst'}:19: WARNING: Provided id 'unknown_id' for needextend does not exist. [needs.extend]"
+        f"{Path(str(app.srcdir)) / 'index.rst'}:19: WARNING: Provided id 'unknown_id' for needextend does not exist. [needs.needextend]"
     ]
 
 
diff --git a/tests/test_needextract.py b/tests/test_needextract.py
index 44ca3e88..d862873d 100644
--- a/tests/test_needextract.py
+++ b/tests/test_needextract.py
@@ -76,10 +76,10 @@ def test_needextract_with_nested_needs(test_app):
     # print(warnings)
     # note these warnings are emitted twice because they are resolved twice: once when first specified and once when copied with needextract
     assert warnings == [
-        'srcdir/index.rst:13: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:33: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:13: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
-        'srcdir/index.rst:33: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecation]',
+        'srcdir/index.rst:13: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:33: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:13: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
+        'srcdir/index.rst:33: WARNING: The [[copy("id")]] syntax in need content is deprecated. Replace with :ndf:`copy("id")` instead. [needs.deprecated]',
     ]
 
     needextract_html = Path(app.outdir, "needextract.html").read_text()
diff --git a/tests/test_needimport.py b/tests/test_needimport.py
index 8a0cd5fb..2f781e86 100644
--- a/tests/test_needimport.py
+++ b/tests/test_needimport.py
@@ -26,7 +26,7 @@ def test_import_json(test_app):
         )
     ).splitlines()
     assert warnings == [
-        "srcdir/subdoc/deprecated_rel_path_import.rst:6: WARNING: Deprecation warning: Relative path must be relative to the current document in future, not to the conf.py location. Use a starting '/', like '/needs.json', to make the path relative to conf.py. [needs]"
+        "srcdir/subdoc/deprecated_rel_path_import.rst:6: WARNING: Deprecation warning: Relative path must be relative to the current document in future, not to the conf.py location. Use a starting '/', like '/needs.json', to make the path relative to conf.py. [needs.deprecated]"
     ]
 
     html = Path(app.outdir, "index.html").read_text()
diff --git a/tests/test_needreport.py b/tests/test_needreport.py
index 36061a51..0e0208e2 100644
--- a/tests/test_needreport.py
+++ b/tests/test_needreport.py
@@ -20,8 +20,8 @@ def test_doc_needarch(test_app):
         .strip()
     ).splitlines()
     assert warnings == [
-        "<srcdir>/index.rst:6: WARNING: No options specified to generate need report [needs.report]",
-        "<srcdir>/index.rst:8: WARNING: Could not load needs report template file <srcdir>/unknown.rst [needs.report]",
+        "<srcdir>/index.rst:6: WARNING: No options specified to generate need report [needs.needreport]",
+        "<srcdir>/index.rst:8: WARNING: Could not load needs report template file <srcdir>/unknown.rst [needs.needreport]",
     ]
 
     html = Path(app.outdir, "index.html").read_text(encoding="utf8")
diff --git a/tests/test_parallel_execution.py b/tests/test_parallel_execution.py
index 9f2bf042..4ae28a10 100644
--- a/tests/test_parallel_execution.py
+++ b/tests/test_parallel_execution.py
@@ -3,6 +3,7 @@ from pathlib import Path
 
 import pytest
 from sphinx.util.console import strip_colors
+from sphinx.util.parallel import parallel_available
 
 
 @pytest.mark.parametrize(
@@ -17,6 +18,7 @@ from sphinx.util.console import strip_colors
     ],
     indirect=True,
 )
+@pytest.mark.skipif(not parallel_available, reason="Parallel execution not supported")
 def test_doc_build_html(test_app):
     app = test_app
     app.build()