Repository: pulp/pulpcore

Issue:
Overwriting existing packages in backend storage can lead to caching issues
If an existing package is re-added to pulp, the default behavior will overwrite the existing file in backing storage. This is typically fine.
- If using Azure Blobstore, the timestamp of the blob is updated (Last-Modified time and ETag).
- Conversely, some CDN's (notably Azure Front Door) use Last-Modified Time as a signal that a file in origin has updated.
- This can lead to poor cache behavior, and in some cases, incomplete downloads as the CDN attempts to resolve disparate content.
- If we set `AZURE_OVERWRITE_FILES` to `false` this partially mitigates the issue (Last-Modified/ETag are unmodified). However, this results in duplicate copies written to storage (with a suffix to differentiate from the original).
- We should have an option that does "nothing" if the uploaded file already exists (don't overwrite, and don't write a new copy).


Discussion:
I was able to reproduce this with a pulp dev environment using minio. Steps to reproduce:

```
pulp rpm repository create --name test
pulp rpm distribution create --name test --repository test --base-path test
pulp rpm content upload --file bear-4.1-1.noarch.rpm
pulp rpm repository content add --repository test --package-href /pulp/api/v3/content/rpm/packages/018e6776-e768-7f2e-bff5-8aeb5bf6e43b/  
pulp rpm publication create --repository test

# now check the Last-Modified header
curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"

# upload the package again
pulp rpm content upload --file bear-4.1-1.noarch.rpm > /dev/null

# check the Last-Modified header again
curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"
```

Here's the output I see:

```
$ curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"
Last-Modified: Fri, 22 Mar 2024 18:40:29 GMT           

$ pulp rpm content upload --file bear-4.1-1.noarch.rpm > /dev/null
Started background task /pulp/api/v3/tasks/018e6782-e5f5-7b25-9e6f-d1e240170130/
.Done.

$ curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"
Last-Modified: Fri, 22 Mar 2024 18:53:35 GMT                                                               
```

So the `Last Modified` header changes when a package is uploaded. I definitely think that reuploading a package shouldn't change its modified timestamp for various reasons.

Like Azure, there is a `AWS_S3_FILE_OVERWRITE` option (defaults to True) but setting it to False makes django-storages create new files in storage with random chars as a suffix which is problematic.
Adding a check [here before save](https://github.com/pulp/pulpcore/blob/a1dd579e5aa831c1dd69b3dc028c6e9108bab06e/pulpcore/plugin/viewsets/content.py#L137
) to try to fetch any existing artifact seems to fix the problem. I think what's happening is that django-storages updates the file in storage regardless of whether the artifact already exists or not.
@daviddavis do you want to open a PR?

Resolving patch:
diff --git a/pulpcore/plugin/viewsets/content.py b/pulpcore/plugin/viewsets/content.py
--- a/pulpcore/plugin/viewsets/content.py
+++ b/pulpcore/plugin/viewsets/content.py
@@ -133,18 +133,20 @@ def init_content_data(self, serializer, request):
             # in the upload code path make sure, the artifact exists, and the 'file'
             # parameter is replaced by 'artifact'
             artifact = Artifact.init_and_validate(task_payload.pop("file"))
+            # if artifact already exists, let's use it
             try:
-                artifact.save()
-            except IntegrityError:
-                # if artifact already exists, let's use it
+                artifact = Artifact.objects.get(
+                    sha256=artifact.sha256, pulp_domain=request.pulp_domain
+                )
+                artifact.touch()
+            except (Artifact.DoesNotExist, DatabaseError):
                 try:
+                    artifact.save()
+                except IntegrityError:
                     artifact = Artifact.objects.get(
                         sha256=artifact.sha256, pulp_domain=request.pulp_domain
                     )
                     artifact.touch()
-                except (Artifact.DoesNotExist, DatabaseError):
-                    # the artifact has since been removed from when we first attempted to save it
-                    artifact.save()
 
             task_payload["artifact"] = ArtifactSerializer(
                 artifact, context={"request": request}


Test patch: