Repository: pulp/pulpcore

Issue:
fs_exporter does not include productid certificate on incremental exports
**Version**
3.18.16

**Describe the bug**
Using fs_exporter to export incrementally a repository that has a productid certificate will not copy the productid certificate.

This happens because we calculate the differences between repository version being export and start_version. As the productid is always the same, then this artifact is not listed as new on the list of differences and is the not included on the export.

We should ensure all artifacts that belong to metadata are exported.

**To Reproduce**
1. Have a repository that contains a productid certificate
2. Generate a complete export of such repository
3. Export an incremental version of this repository using fs_exporter
4. Check that the productid certificate of the repository is missing from the export

**Expected behavior**
All metadata files to be exported, regardless if the export is complete or incremental

**Additional context**
https://bugzilla.redhat.com/show_bug.cgi?id=2212523



Discussion:
Copying over my comment from the PR:

----

I would consider this either a Katello or a documentation bug then.  This is the way this feature was described when it was requested / added:

https://github.com/pulp/pulpcore/issues/3413

> Idea is
> 
> *  User has already exported Rhel - 7 (using the fs exporter) - full export
>  *  User consumes content from the export
>  *  Several days later the user now wants the new content for RHEL 7. Instead of exporting 50GB worth of content again the user's export should only consist of rpms that got added after the start version and latest metadata
>   * User should be able to **copy the contents of this export over the regular rhel export directory** and use that to sync repodata.

From Pulp's perspective I don't think there is a bug here.
So there is / was some misunderstandings around the original design.  The intention was for the incremental filesystem export to also be syncable on its own, which means it needs to have all the metadata, even if it didn't change, whereas the current incremental implementation only includes artifacts that changed and expects the user to copy it on top of an existing filesystem export.

If copying isn't an option, then we could change the implementation to export metadata as well.  Copying Grant's suggestion:

>Maybe somewhere around here https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/export.py#L146-L155 we could look for for all PublishedMetadata associated with the passed-in publication, and force-add them to relative_path_to_artifacts - I think that will address the need, without exposing pulpcore to "knowing things about plugins Core Was Not Meant to Know"

Resolving patch:
diff --git a/pulpcore/app/tasks/export.py b/pulpcore/app/tasks/export.py
--- a/pulpcore/app/tasks/export.py
+++ b/pulpcore/app/tasks/export.py
@@ -143,12 +143,17 @@ def _export_publication_to_file_system(
             for ca in content_artifacts.select_related("artifact").iterator()
         }
 
+    publication_metadata_paths = set(
+        publication.published_metadata.values_list("relative_path", flat=True)
+    )
     for pa in publication.published_artifact.select_related(
         "content_artifact", "content_artifact__artifact"
     ).iterator():
         # Artifact isn't guaranteed to be present
         if pa.content_artifact.artifact and (
-            start_repo_version is None or pa.content_artifact.pk in difference_content_artifacts
+            start_repo_version is None
+            or pa.relative_path in publication_metadata_paths
+            or pa.content_artifact.pk in difference_content_artifacts
         ):
             relative_path_to_artifacts[pa.relative_path] = pa.content_artifact.artifact
 


Test patch: