Repository: hydroshare/hydroshare

Issue:
Update deposit into Crossref is failing when the update XML is the same as existing
@ClaraCogswell received 16 email notifications for a resource that is:

> "Request to update CrossRef deposit for published resource DOI {act_doi} is still in a pending state."

This is generated here:
https://github.com/hydroshare/hydroshare/blob/2.12/hs_core/tasks.py#L377-L379
because the resource DOI is in `_update_pending` state and it will never get out of it. So emails will continue every hour forever.

We should revise this behavior.
I propose we also route this type of task email to the dev team.

Example resource:
https://www.hydroshare.org/resource/493e0ad05c2d45199427cc41a6c76de0/

**Additional information**
HS v2.12



Discussion:
perhaps related to https://github.com/hydroshare/hydroshare/issues/5320

we realized this was due to running `update_crossref_metadata_deposit 493e0ad05c2d45199427cc41a6c76de0` which created a duplicate request with Crossref. The temporary solution was 
```
>>> from hs_core.models import BaseResource
>>> res = BaseResource.objects.get(short_id='493e0ad05c2d45199427cc41a6c76de0')
>>> res
<BaseResource: Notebooks for combining the National Water Model results/inputs with observations from SNOTEL and MODIS at SNOTEL sites>
>>> res.doi
'https://doi.org/10.4211/hs.493e0ad05c2d45199427cc41a6c76de0update_pending'
>>> res.doi = 'https://doi.org/10.4211/hs.493e0ad05c2d45199427cc41a6c76de0'
>>> res.save()
```
@devincowan It seems every time we make a crossref deposit request, we need to make another request to crossref to check the `failure_count` and accordingly set the doi status.  We can discuss this more in slack. 
@pkdash that sounds good.
Another option to consider:
right now it looks like we `update_crossref_deposit()` in the following locations:

1. [CoreMetadata.create_element()](https://github.com/hydroshare/hydroshare/blob/2.12/hs_core/models.py#L4975)
2. [CoreMetadata.update_element()](https://github.com/hydroshare/hydroshare/blob/2.12/hs_core/models.py#L4994)
3. [CoreMetadata.delete_element()](https://github.com/hydroshare/hydroshare/blob/2.12/hs_core/models.py#L5007)

Instead of adding and additional request to Crossref, we could detect change to `fundingagency` or `description` in these 3 places and only `update_crossref_deposit()` if the values did in fact change and thus prevent duplicate deposits.

Though we would have to test what happens if one makes a change to a published resource, saves, and then changes _back_ to the original...

@devincowan 

> Though we would have to test what happens if one makes a change to a published resource, saves, and then changes back to the original..

It seems Crossref using only the `<timestamp>` when checking for duplicate submission - so the above scenario should not be a problem in updating metadata. Ref: https://www.crossref.org/documentation/register-maintain-records/maintaining-your-metadata/updating-your-metadata/
@pkdash that is mighty interesting, good find!
So if they are only using the timestamp for checking duplicate submissions, it had me wondering why our `update_crossref_metadata_deposit 493e0ad05c2d45199427cc41a6c76de0` created a submission with the same timestamp as the previous submission. 

Here's where we get the timestamp:
https://github.com/hydroshare/hydroshare/blob/2.12/hs_core/models.py#L4026-L4027
Seems like our fix might be as simple as changing this timestamp to be based on `.now()` (the time of the XML generation) instead of `self.updated`
@pkdash see Jeff's schema for the timestamp:
https://github.com/hydroshare/hs_doi_deposit_metadata/blob/main/docs/HydroShare_Crossref_Deposit_Metadata.md#head

> timestamp: The time at which the deposit metadata was created with second resolution.


Resolving patch:
diff --git a/hs_core/models.py b/hs_core/models.py
--- a/hs_core/models.py
+++ b/hs_core/models.py
@@ -4023,8 +4023,7 @@ def create_date_node(date, date_type):
         # create the head sub element
         head_node = etree.SubElement(ROOT, 'head')
         etree.SubElement(head_node, 'doi_batch_id').text = self.short_id
-        etree.SubElement(head_node, 'timestamp').text = arrow.get(self.updated)\
-            .format("YYYYMMDDHHmmss")
+        etree.SubElement(head_node, 'timestamp').text = arrow.now().format("YYYYMMDDHHmmss")
         depositor_node = etree.SubElement(head_node, 'depositor')
         etree.SubElement(depositor_node, 'depositor_name').text = 'HydroShare'
         etree.SubElement(depositor_node, 'email_address').text = settings.DEFAULT_SUPPORT_EMAIL


Test patch:
diff --git a/hs_core/tests/api/native/test_publish_resource.py b/hs_core/tests/api/native/test_publish_resource.py
--- a/hs_core/tests/api/native/test_publish_resource.py
+++ b/hs_core/tests/api/native/test_publish_resource.py
@@ -2,6 +2,7 @@
 import unittest
 
 import arrow
+from freezegun import freeze_time
 from django.conf import settings
 from django.contrib.auth.models import Group, User
 from django.test import TestCase
@@ -107,12 +108,16 @@ def test_crossref_deposit_xml(self):
         if not hasattr(settings, 'DEFAULT_SUPPORT_EMAIL'):
             settings.DEFAULT_SUPPORT_EMAIL = "help@cuahsi.org"
 
+        freeze = arrow.now().format("YYYY-MM-DD HH:mm:ss")
+        freezer = freeze_time(freeze)
+        freezer.start()
+
         # set doi - simulating published resource
         self.res.doi = get_resource_doi(self.res.short_id)
         self.res.save()
         crossref_xml = self.res.get_crossref_deposit_xml()
         crossref_xml = crossref_xml.strip()
-        timestamp = arrow.get(self.res.updated).format("YYYYMMDDHHmmss")
+        timestamp = arrow.now().format("YYYYMMDDHHmmss")
         res_id = self.res.short_id
         created_date = self.res.created
         updated_date = self.res.updated
@@ -126,6 +131,9 @@ def test_crossref_deposit_xml(self):
                                                        published_date=published_date, site_url=site_url,
                                                        support_email=support_email)
         expected_xml = expected_xml.strip()
+
+        freezer.stop()
+
         self.assertTrue(len(crossref_xml) == len(expected_xml))
         match_ratio = difflib.SequenceMatcher(None, crossref_xml.splitlines(), expected_xml.splitlines()).ratio()
         self.assertTrue(match_ratio == 1.0, msg="crossref xml is not as expected")