Repository: hydroshare/hydroshare

Issue:
json endpoints broken for resources in publishing review
**Description of the bug**
The `hsapi2/resource/{id}/json/` endpoint (and other places that use hsmodels) are currently broken due to a new date type (review_started) that is not a supported enumeration of http://purl.org/dc/terms/.

Steps to reproduce the bug:
1. Go to https://hydroshare.org/hsapi2
2. Send a GET request for a resource in review (31b56632213345db889a9eee981619f6)
3. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional information**
The quick fix is probably to update the rdf_triples method in the Date AbstractMetaDataElement implementation to not create a triple for dates with the new type. 
```
[17/Mar/2023 02:10:12] ERROR [django.request:224] Internal Server Error: /hsapi2/resource/31b56632213345db889a9eee981619f6/json/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/decorators.py", line 50, in handler
    return func(*args, **kwargs)
  File "/hydroshare/hs_rest_api2/views/metadata.py", line 28, in resource_metadata_json
    md_json = resource_metadata_json_loads(resource)
  File "/hydroshare/hs_rest_api2/metadata.py", line 73, in resource_metadata_json_loads
    return json.loads(resource_metadata(resource).json())
  File "/hydroshare/hs_rest_api2/metadata.py", line 66, in resource_metadata
    return load_metadata_from_file(istorage, file_with_path)
  File "/hydroshare/hs_rest_api2/metadata.py", line 47, in load_metadata_from_file
    metadata = load_rdf(f.read())
  File "/usr/local/lib/python3.9/site-packages/hsmodels/schemas/__init__.py", line 71, in load_rdf
    rdf_metadata = _parse(schema, g)
  File "/usr/local/lib/python3.9/site-packages/hsmodels/schemas/__init__.py", line 181, in _parse
    parsed_class = _parse(clazz, metadata_graph, value)
  File "/usr/local/lib/python3.9/site-packages/hsmodels/schemas/__init__.py", line 199, in _parse
    instance = schema(**kwargs, rdf_subject=subject)
  File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for DateInRDF
type
  value is not a valid enumeration member; permitted: 'http://purl.org/dc/terms/modified', 'http://purl.org/dc/terms/created', 'http://purl.org/dc/terms/valid', 'http://purl.org/dc/terms/available', 'http://purl.org/dc/terms/published' (type=type_error.enum; enum_values=[<DateType.modified: 'http://purl.org/dc/terms/modified'>, <DateType.created: 'http://purl.org/dc/terms/created'>, <DateType.valid: 'http://purl.org/dc/terms/valid'>, <DateType.available: 'http://purl.org/dc/terms/available'>, <DateType.published: 'http://purl.org/dc/terms/published'>])
```



Discussion:
@sblack-usu I think `published` is also not a supported enumeration in http://purl.org/dc/terms/
But even so, we use it:
https://github.com/hydroshare/hsmodels/blob/main/hsmodels/schemas/enums.py#L90
https://github.com/hydroshare/hydroshare/blob/master/hs_core/models.py#L873
am I wrong?

I wanted to understand how much room there is for flexibility in the namespaces so I checked:
https://rdflib.readthedocs.io/en/stable/_modules/rdflib/namespace/_DCTERMS.html
("published" is not a term)

Oh realized we are using rdflib v5
https://github.com/hydroshare/hs_docker_base/blob/develop/requirements.txt#L147
So checked here:
https://github.com/RDFLib/rdflib/blob/5.0.0/rdflib/namespace.py#L261

So my question I guess is: why doesnt this hsapi2 endpoint fail for GET on a _published_ resource? :)
I verified that the following change on hsmodels "patches" the hsapi2 endpoint:
https://github.com/hydroshare/hsmodels/compare/main...rdf_test

Of course, we should ideally handle the "published" and the "review_pending" terms differently (instead of falsely implying that they are DCTERMS when they aren't). I guess we should add these to HSTERMS? Or what do you recommend @sblack-usu ? 

I will open a [PR](https://github.com/hydroshare/hsmodels/pull/41) on hsmodels for the above patch, if that's the approach you'd like for the time being...
Our plan is to make "review_pending" and "published" HSTERMS instead of DCTERMS
@devincowan - that’s the right way to go because we don’t control the DCTERMS namespace and can’t assert new terms in that namespace. 

Resolving patch:
diff --git a/hs_core/models.py b/hs_core/models.py
--- a/hs_core/models.py
+++ b/hs_core/models.py
@@ -864,14 +864,17 @@ def ingest_rdf(cls, graph, subject, content_object):
 class Date(AbstractMetaDataElement):
     """Define Date metadata model."""
 
-    DATE_TYPE_CHOICES = (
+    DC_DATE_TYPE_CHOICES = (
         ('created', 'Created'),
         ('modified', 'Modified'),
         ('valid', 'Valid'),
-        ('available', 'Available'),
-        ('review_started', 'Review Started'),
+        ('available', 'Available')
+    )
+    HS_DATE_TYPE_CHOICES = (
+        ('reviewStarted', 'Review Started'),
         ('published', 'Published')
     )
+    DATE_TYPE_CHOICES = DC_DATE_TYPE_CHOICES + HS_DATE_TYPE_CHOICES
 
     term = 'Date'
     type = models.CharField(max_length=20, choices=DATE_TYPE_CHOICES)
@@ -893,7 +896,10 @@ class Meta:
     def rdf_triples(self, subject, graph):
         date_node = BNode()
         graph.add((subject, self.get_class_term(), date_node))
-        graph.add((date_node, RDF.type, getattr(DCTERMS, self.type)))
+        if self.type in [inner[0] for inner in self.DC_DATE_TYPE_CHOICES]:
+            graph.add((date_node, RDF.type, getattr(DCTERMS, self.type)))
+        else:
+            graph.add((date_node, RDF.type, getattr(HSTERMS, self.type)))
         graph.add((date_node, RDF.value, Literal(self.start_date.isoformat())))
 
     @classmethod


Test patch:
diff --git a/hs_core/tests/data/d6c7a5744920404f8aceaf3c7774596e/d6c7a5744920404f8aceaf3c7774596e/data/resourcemetadata.xml b/hs_core/tests/data/d6c7a5744920404f8aceaf3c7774596e/d6c7a5744920404f8aceaf3c7774596e/data/resourcemetadata.xml
--- a/hs_core/tests/data/d6c7a5744920404f8aceaf3c7774596e/d6c7a5744920404f8aceaf3c7774596e/data/resourcemetadata.xml
+++ b/hs_core/tests/data/d6c7a5744920404f8aceaf3c7774596e/d6c7a5744920404f8aceaf3c7774596e/data/resourcemetadata.xml
@@ -30,6 +30,11 @@
         <rdf:value>2022-04-12T17:43:07.468505+00:00</rdf:value>
       </dcterms:modified>
     </dc:date>
+    <dc:date>
+      <hsterms:reviewStarted>
+        <rdf:value>2022-04-13T17:43:07.468505+00:00</rdf:value>
+      </hsterms:reviewStarted>
+    </dc:date>
     <dc:rights>
       <rdf:Description>
         <hsterms:rightsStatement>This resource is shared under the Creative Commons Attribution CC BY.</hsterms:rightsStatement>
diff --git a/hs_core/tests/data/test_resource_metadata_files/resource/resourcemetadata.xml b/hs_core/tests/data/test_resource_metadata_files/resource/resourcemetadata.xml
--- a/hs_core/tests/data/test_resource_metadata_files/resource/resourcemetadata.xml
+++ b/hs_core/tests/data/test_resource_metadata_files/resource/resourcemetadata.xml
@@ -86,6 +86,11 @@
         <rdf:value>2020-07-15T23:12:19.334831+00:00</rdf:value>
       </dcterms:modified>
     </dc:date>
+    <dc:date>
+      <hsterms:reviewStarted>
+        <rdf:value>2020-07-16T23:12:19.334831+00:00</rdf:value>
+      </hsterms:reviewStarted>
+    </dc:date>
     <dc:subject>key2</dc:subject>
     <dc:description>
       <rdf:Description>
diff --git a/hs_rest_api2/tests/data/json/resource.json b/hs_rest_api2/tests/data/json/resource.json
--- a/hs_rest_api2/tests/data/json/resource.json
+++ b/hs_rest_api2/tests/data/json/resource.json
@@ -80,6 +80,7 @@
   },
   "created": "2020-07-09T19:12:21.354703+00:00",
   "modified": "2020-11-13T19:40:57.276064+00:00",
+  "review_started": "2020-07-16T23:12:19.334831+00:00",
   "awards": [
     {
       "funding_agency_name": "agency1",