Repository: oscal-compass/compliance-trestle

Issue:
x-trestle-add-props duplicates props on every ssp-assemble
## Describe the bug

Calling `trestle author ssp-assemble` when there have been no changes results in duplicating every prop added with `x-trestle-add-props`

## To Reproduce

Steps to reproduce the behavior:

1. `trestle author ssp-generate -p sc81-profile -o control-statements -iap` (profile pasted at bottom)
1. Add a prop to `x-trestle-add-props` in `control-statements/sc-8.1.md`:
    ```
    x-trestle-add-props:
      - name: test-property
        value: bug reproduction
    ```

1. `trestle author ssp-assemble -o bug-reproduction -m control-statements`
1. Repeat assemble step

## Expected behavior

The second assemble step results in a message about how the ssp has not changed and will not be updated

## Actual behavior

The second assemble step updates the `metadata.last-modified` timestamp and duplicates the `test-property` prop on the SC-8.1 control.

Additional calls to ssp-assemble result in more and more props being added to the control.

## Screenshots / Logs.

sc81-profile:
```json
{
  "profile": {
    "uuid": "83be30e4-6c88-40e0-9927-286ff0e43d59",
    "metadata": {
      "title": "SC-8.1 profile",
      "last-modified": "2024-06-24T19:35:43.973109+00:00",
      "version": "0.0.1",
      "oscal-version": "1.1.2"
    },
    "imports": [
      {
        "href": "https://raw.githubusercontent.com/usnistgov/oscal-content/main/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_catalog.json",
        "include-controls": [
          {
            "with-ids": [
              "sc-8.1"
            ]
          }
        ]
      }
    ]
  }
}
```

## Environment

- OS: python:3.11.11-slim docker image
- Python version: 3.11.11
- Installed packages:

<details><summary>pip list</summary>

```
appuser@ff0deeadc25b:/app/docs$ pip list
WARNING: The directory '/nonexistent/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Package                  Version
------------------------ ----------
annotated-types          0.7.0
anyio                    4.8.0
argcomplete              3.5.3
attrs                    24.3.0
bcrypt                   4.2.1
black                    24.10.0
certifi                  2024.12.14
cffi                     1.17.1
charset-normalizer       3.4.1
click                    8.1.8
cmarkgfm                 2024.1.14
compliance-trestle       3.6.0
compliance-trestle-gsa   0.0.1
cryptography             43.0.3
datamodel-code-generator 0.25.3
defusedxml               0.7.1
dnspython                2.7.0
email_validator          2.2.0
et_xmlfile               2.0.0
furl                     2.1.3
genson                   1.3.0
h11                      0.14.0
httpcore                 1.0.7
httpx                    0.28.1
idna                     3.10
ilcli                    0.3.2
importlib_resources      6.5.2
inflect                  5.6.2
isort                    5.13.2
Jinja2                   3.1.4
MarkupSafe               3.0.2
mypy-extensions          1.0.0
openpyxl                 3.1.5
orderedmultidict         1.0.1
orjson                   3.10.14
packaging                24.2
paramiko                 3.4.0
pathspec                 0.12.1
pip                      24.0
platformdirs             4.3.6
pycparser                2.22
pydantic                 2.10.5
pydantic_core            2.27.2
PyNaCl                   1.5.0
python-dotenv            1.0.1
python-frontmatter       1.1.0
PyYAML                   6.0.2
requests                 2.32.3
ruamel.yaml              0.18.10
ruamel.yaml.clib         0.2.12
setuptools               65.5.1
six                      1.17.0
sniffio                  1.3.1
typing_extensions        4.12.2
urllib3                  2.3.0
wheel                    0.45.1
```
</details>

Hints from issue comments:

Solution patch:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 049d3010..8ec997d7 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -7,7 +7,7 @@ To contribute code or documentation, please submit a [pull request](https://gith
 
 A good way to familiarize yourself with the codebase and contribution process is
 to look for and tackle low-hanging fruit in the [issue tracker](https://github.com/oscal-compass/compliance-trestle/issues).
-Before embarking on a more ambitious contribution, please quickly [get in touch](https://oscal-compass.github.io/compliance-trestle/maintainers/) with us.
+Before embarking on a more ambitious contribution, please quickly [get in touch](https://oscal-compass.github.io/compliance-trestle/latest/contributing/maintainers/) with us.
 
 **Note: We appreciate your effort, and want to avoid a situation where a contribution
 requires extensive rework (by you or by us), sits in backlog for a long time, or
@@ -32,7 +32,7 @@ review to indicate acceptance.
 
 A change requires LGTMs from at least two reviewers. One of the reviewers must be a [`CODEOWNER`](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners).
 
-For a list of the maintainers (also codeowners), see the [maintainers](https://oscal-compass.github.io/compliance-trestle/maintainers/) page.
+For a list of the maintainers (also codeowners), see the [maintainers](https://oscal-compass.github.io/compliance-trestle/latest/contributing/maintainers/)
 
 ### Trestle updating, testing and release logistics
 
@@ -88,7 +88,7 @@ The devops process does not _strictly_ enforce typing, however, the expectation
 commits with a focus on quality over quantity (e.g. don't add `Any` everywhere just to meet coverage requirements).
 Python typing of functions is an active work in progress.
 
-`mkbuild` is used to generate the [trestle documenation site](https://oscal-compass.github.io/compliance-trestle). The `mkbuild`
+`mkbuild` is used to generate the [trestle documenation site](https://oscal-compass.github.io/compliance-trestle/latest). The `mkbuild`
 website includes an API reference section generated from the code. Docstrings within the code are expected to follow
 [google style docstrings](https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html).
 
@@ -116,7 +116,7 @@ e.g.
 
 We have tried to make it as easy as possible to make contributions. This
 applies to how we handle the legal aspects of contribution. We use the
-same approach - the [Developer's Certificate of Origin 1.1 (DCO)](https://oscal-compass.github.io/compliance-trestle/contributing/DCO/) - that the Linux® Kernel [community](https://developercertificate.org/)
+same approach - the [Developer's Certificate of Origin 1.1 (DCO)](https://oscal-compass.github.io/compliance-trestle/latest/contributing/DCO/) - that the Linux® Kernel [community](https://developercertificate.org/)
 uses to manage code contributions.
 
 We simply ask that when submitting a patch for review, the developer
diff --git a/README.md b/README.md
index 731f01d1..be8b71a4 100644
--- a/README.md
+++ b/README.md
@@ -64,11 +64,11 @@ Users needing to import XML OSCAL artifacts are recommended to look at NIST's XM
 
 Trestle runs on almost all Python platforms (e.g. Linux, Mac, Windows), is available on PyPi and can be easily installed via pip. It is under active development and new releases are made available regularly.\
 To install run: `pip install compliance-trestle`\
-See [Install trestle in a python virtual environment](https://oscal-compass.github.io/compliance-trestle/python_trestle_setup/) for the full installation guide.
+See [Install trestle in a python virtual environment](https://oscal-compass.github.io/compliance-trestle/latest/installation/) for the full installation guide.
 
 ## Complete documentation and tutorials
 
-Complete documentation, tutorials, and background on compliance can be found [here](https://oscal-compass.github.io/compliance-trestle).
+Complete documentation, tutorials, and background on compliance can be found [here](https://oscal-compass.github.io/compliance-trestle/latest).
 
 ## Agile Authoring
 
@@ -101,7 +101,7 @@ Please refer to the community [README](https://github.com/oscal-compass/communit
 
 ## Contributing to Trestle
 
-Our project welcomes external contributions. Please consult [contributing](https://oscal-compass.github.io/compliance-trestle/contributing/mkdocs_contributing/) to get started.
+Our project welcomes external contributions. Please consult [contributing](https://oscal-compass.github.io/compliance-trestle/latest/contributing/mkdocs_contributing/) to get started.
 
 ## Code of Conduct
 
diff --git a/docs/tutorials/Transformers_and_Tasks/OCP4_CIS_profile_to_oscal_cd.md b/docs/tutorials/Transformers_and_Tasks/OCP4_CIS_profile_to_oscal_cd.md
index 11964749..ceb12f7f 100644
--- a/docs/tutorials/Transformers_and_Tasks/OCP4_CIS_profile_to_oscal_cd.md
+++ b/docs/tutorials/Transformers_and_Tasks/OCP4_CIS_profile_to_oscal_cd.md
@@ -17,7 +17,7 @@ The second is a one-command transformation from `.profile` to `OSCAL.json`.
 
 ## Step 1: Install trestle in a Python virtual environment
 
-Follow the instructions [here](https://oscal-compass.github.io/compliance-trestle/python_trestle_setup/) to install trestle in a virtual environment.
+Follow the instructions [here](https://oscal-compass.github.io/compliance-trestle/latest/installation/) to install trestle in a virtual environment.
 
 ## Step 2: Transform profile data (CIS benchmarks)
 
diff --git a/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd.md b/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd.md
index 13aa4898..90c16d22 100644
--- a/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd.md
+++ b/docs/tutorials/Transformers_and_Tasks/csv_to_oscal_cd.md
@@ -169,7 +169,7 @@ The below table represents the expectations of trestle task `csv-to-oscal-cd` fo
 
 ## *Step 1: Install trestle in a Python virtual environment*
 
-Follow the instructions [here](https://oscal-compass.github.io/compliance-trestle/python_trestle_setup/) to install trestle in a virtual environment.
+Follow the instructions [here](https://oscal-compass.github.io/compliance-trestle/latest/installation/) to install trestle in a virtual environment.
 
 ## *Step 2: Transform profile data (CIS benchmarks)*
 
diff --git a/docs/tutorials/Trestle_authoring/ssp_profile_catalog_authoring.md b/docs/tutorials/Trestle_authoring/ssp_profile_catalog_authoring.md
index 3a888169..ff3185d3 100644
--- a/docs/tutorials/Trestle_authoring/ssp_profile_catalog_authoring.md
+++ b/docs/tutorials/Trestle_authoring/ssp_profile_catalog_authoring.md
@@ -321,7 +321,7 @@ Access control policy and procedures address the controls in the AC family that
 <!-- "## Part" parts are new subparts added into the existing top-level statement part with that label. -->
 <!-- Subparts may be added with nested hash levels of the form ### My Subpart Name -->
 <!-- underneath the parent ## Control or ## Part being added -->
-<!-- See https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring for guidance. -->
+<!-- See https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/ for guidance. -->
 
 ## Control Implementation Guidance
 
diff --git a/docs/tutorials/Trestle_authoring/trestle_author.md b/docs/tutorials/Trestle_authoring/trestle_author.md
index 9ddffa33..3340861f 100644
--- a/docs/tutorials/Trestle_authoring/trestle_author.md
+++ b/docs/tutorials/Trestle_authoring/trestle_author.md
@@ -690,7 +690,7 @@ CLI evocation:
 
 > trestle author catalog-assemble
 
-The `catalog` author commands allow you to convert a control catalog to markdown and edit its control statement, then assemble markdown back into an OSCAL catalog with the modifications to the statement.  Items in the statement may be edited or added.  For more details on its usage please see [the catalog authoring tutorial](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring).
+The `catalog` author commands allow you to convert a control catalog to markdown and edit its control statement, then assemble markdown back into an OSCAL catalog with the modifications to the statement.  Items in the statement may be edited or added.  For more details on its usage please see [the catalog authoring tutorial](https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/).
 
 ### Profile authoring
 
@@ -704,7 +704,7 @@ CLI evocation:
 
 > trestle author profile-assemble
 
-The `profile` author commands allow you to edit additions made by a profile to its imported controls that end up in the final resolved profile catalog.  Only the additions may be edited or added to the generated markdown control files - and those additions can then be assembled into a new version of the original profile, with those additions.  For more details on its usage please see [the profile authoring tutorial](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring).
+The `profile` author commands allow you to edit additions made by a profile to its imported controls that end up in the final resolved profile catalog.  Only the additions may be edited or added to the generated markdown control files - and those additions can then be assembled into a new version of the original profile, with those additions.  For more details on its usage please see [the profile authoring tutorial](https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/).
 
 ### Profile generation with inheritance
 
@@ -719,7 +719,7 @@ All components must have exported provided statements, no exported responsibilit
 
 As with the other related author commands, if an existing destination file already exists, it is not updated if no changes would be made.
 
-For more details on its usage please see [the ssp-filter tutorial](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring).
+For more details on its usage please see [the ssp-filter tutorial](https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/).
 
 ### SSP authoring
 
@@ -735,7 +735,7 @@ CLI evocation:
 
 The `ssp-generate` sub-command creates a partial SSP (System Security Plan) from a profile and optional yaml header file.  `ssp-assemble` can then assemble the markdown files into a single json SSP file.
 
-For more details on its usage please see [the ssp authoring tutorial](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring).
+For more details on its usage please see [the ssp authoring tutorial](https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/).
 
 ### SSP Content Filtering
 
@@ -757,6 +757,6 @@ You may filter by a combination of a profile, list of component names, implement
 
 As with the other related author commands, if an existing destination file already exists, it is not updated if no changes would be made.
 
-For more details on its usage please see [the ssp-filter tutorial](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring).
+For more details on its usage please see [the ssp-filter tutorial](https://oscal-compass.github.io/compliance-trestle/latest/tutorials/Trestle_authoring/ssp_profile_catalog_authoring/).
 
 </details>
diff --git a/trestle/core/catalog/catalog_reader.py b/trestle/core/catalog/catalog_reader.py
index 4ed76d13..2e951be0 100644
--- a/trestle/core/catalog/catalog_reader.py
+++ b/trestle/core/catalog/catalog_reader.py
@@ -349,7 +349,7 @@ class CatalogReader():
         # add the props at control level
         if props:
             imp_req.props = as_list(imp_req.props)
-            imp_req.props.extend(props)
+            ControlInterface.reconcile_props(imp_req, props)
 
         # add the props at the part level
         for label, part_id in control_part_id_map.items():
@@ -359,7 +359,7 @@ class CatalogReader():
             for statement in as_list(imp_req.statements):
                 if statement.statement_id == part_id:
                     statement.props = as_list(statement.props)
-                    statement.props.extend(props)
+                    ControlInterface.reconcile_props(statement, props)
 
     @staticmethod
     def _update_ssp_with_md_header(
diff --git a/trestle/core/control_interface.py b/trestle/core/control_interface.py
index ab3c4c95..69611ae0 100644
--- a/trestle/core/control_interface.py
+++ b/trestle/core/control_interface.py
@@ -1121,6 +1121,18 @@ class ControlInterface:
         prop = ControlInterface._status_as_prop(status)
         ControlInterface._replace_prop(item, prop)
 
+    @staticmethod
+    def reconcile_props(item: TypeWithProps, props: List[common.Property]) -> None:
+        """Add properties to an item with properties while replacing existing."""
+        names = [prop.name for prop in as_list(item.props)]
+        item.props = as_list(item.props)
+        for prop in props:
+            if prop.name in names:
+                index = names.index(prop.name)
+                item.props[index] = prop
+            else:
+                item.props.append(prop)
+
     @staticmethod
     def _copy_status_in_props(dest: TypeWithProps, src: TypeWithProps) -> None:
         """Copy status in props from one object to another."""


Test patch:
diff --git a/tests/trestle/core/commands/author/ssp_test.py b/tests/trestle/core/commands/author/ssp_test.py
index ec4df8d7..0df5ce34 100644
--- a/tests/trestle/core/commands/author/ssp_test.py
+++ b/tests/trestle/core/commands/author/ssp_test.py
@@ -1160,14 +1160,19 @@ def test_ssp_gen_and_assemble_add_props(tmp_trestle_dir: pathlib.Path) -> None:
     impl_reqs = assem_ssp.control_implementation.implemented_requirements
     impl_req = next((i_req for i_req in impl_reqs if i_req.control_id == 'ac-1'), None)
     assert len(impl_req.props) == 1
-    assert impl_req.props[0].name == 'prop_with_ns'
-    assert impl_req.props[0].value == 'prop with ns'
-    assert impl_req.props[0].ns == 'https://my_new_namespace'
+    assert impl_req.props[0].name == 'prop_with_ns'  # type: ignore
+    assert impl_req.props[0].value == 'prop with ns'  # type: ignore
+    assert impl_req.props[0].ns == 'https://my_new_namespace'  # type: ignore
 
     smt_a = next((smt for smt in impl_req.statements if smt.statement_id == 'ac-1_smt.a'), None)
     assert len(smt_a.props) == 1
-    assert smt_a.props[0].name == 'smt_prop'
-    assert smt_a.props[0].value == 'smt prop'
+    assert smt_a.props[0].name == 'smt_prop'  # type: ignore
+    assert smt_a.props[0].value == 'smt prop'  # type: ignore
+
+    # Run again and check that there is no change
+    assert ssp_assemble._run(args) == 0
+    assem_ssp_2, _ = ModelUtils.load_model_for_class(tmp_trestle_dir, ssp_name, ossp.SystemSecurityPlan)
+    assert assem_ssp_2.metadata.last_modified == assem_ssp.metadata.last_modified
 
 
 def test_ssp_gen_and_assemble_implementation_parts(tmp_trestle_dir: pathlib.Path, monkeypatch: MonkeyPatch) -> None: