Repository: Informasjonsforvaltning/fdk-rdf-parser

Issue:
Legg til støtte for Definisjon(euvoc:XlNote) fra skos-ap-no v.2.0.3
se https://data.norge.no/specification/skos-ap-no-begrep#Definisjon

predikater fra gammel skos-ap-no og ekvivalent i ny:
- `rdfs:label` -> `rdf:value`
- `skos:scopeNote` -> flyttes opp til selve begrepet og fjernes fra Definisjon
- `dct:audience` -> `dct:audience` beholdes men bruker ny kodeliste https://data.norge.no/vocabulary/audience-type
- `skosno:forholdTilKilde` -> `skosno:relationshipWithSource` og bruker ny kodeliste https://data.norge.no/vocabulary/relationship-with-source-type
- `skosno:omfang` -> endres til `skosno:valueRange` og flyttes opp til selve begrepet og fjernes fra Definisjon
- `dct:source` -> `dct:source` beholdes men endres til direkte angivelse av uri og fjerner det medfølgende tekstfeltet. Avventer i tillegg støtte for angivelse vha rdf:List
- `dct:modified` -> fjernes

Discussion:
github-actions[bot]: ```
Warning: poetry-plugin-export will not be installed by default in a future version of Poetry.
In order to avoid a breaking change and make your automation forward-compatible, please install poetry-plugin-export explicitly. See https://python-poetry.org/docs/plugins/#using-plugins for details on how to install a plugin.
To disable this warning run 'poetry config warnings.export false'.
The `--dev` option is deprecated, use the `--with dev` notation instead.
+==============================================================================+

                                   /$$$$$$            /$$
                                  /$$__  $$          | $$
               /$$$$$$$  /$$$$$$ | $$  \__//$$$$$$  /$$$$$$   /$$   /$$
              /$$_____/ |____  $$| $$$$   /$$__  $$|_  $$_/  | $$  | $$
             |  $$$$$$   /$$$$$$$| $$_/  | $$$$$$$$  | $$    | $$  | $$
              \____  $$ /$$__  $$| $$    | $$_____/  | $$ /$$| $$  | $$
              /$$$$$$$/|  $$$$$$$| $$    |  $$$$$$$  |  $$$$/|  $$$$$$$
             |_______/  \_______/|__/     \_______/   \___/   \____  $$
                                                              /$$  | $$
                                                             |  $$$$$$/
      by safetycli.com                                        \______/

+==============================================================================+

 REPORT 

  Safety is using PyUp's free open-source vulnerability database. This
data is 30 days old and limited. 
  For real-time enhanced vulnerability data, fix recommendations, severity
reporting, cybersecurity support, team and project policy management and more
sign up at https://pyup.io or email sales@pyup.io

  Safety v3.1.0 is scanning for Vulnerabilities...
  Scanning dependencies in your files:

  -> /tmp/tmp_ymo0e4j

  Using open-source vulnerability database
  Found and scanned 72 packages
  Timestamp 2024-05-03 06:18:57
  0 vulnerabilities reported
  0 vulnerabilities ignored

+==============================================================================+

 No known security vulnerabilities found.

+==============================================================================+

  Safety is using PyUp's free open-source vulnerability database. This
data is 30 days old and limited. 
  For real-time enhanced vulnerability data, fix recommendations, severity
reporting, cybersecurity support, team and project policy management and more
sign up at https://pyup.io or email sales@pyup.io

+==============================================================================+
```
<!-- Sticky Pull Request CommentSafety result -->

Patch:
diff --git a/src/fdk_rdf_parser/classes/concept.py b/src/fdk_rdf_parser/classes/concept.py
index c88371a..a4660b5 100644
--- a/src/fdk_rdf_parser/classes/concept.py
+++ b/src/fdk_rdf_parser/classes/concept.py
@@ -29,10 +29,10 @@ class TextAndURI:
 @dataclass
 class Definition:
     text: Optional[Dict[str, str]] = None
-    remark: Optional[Dict[str, str]] = None
+    remark: Optional[Dict[str, str]] = None  # Deprecated
     targetGroup: Optional[str] = None
     sourceRelationship: Optional[str] = None
-    range: Optional[TextAndURI] = None
+    range: Optional[TextAndURI] = None  # Deprecated
     sources: Optional[List[TextAndURI]] = None
     lastUpdated: Optional[str] = None
 
@@ -91,4 +91,6 @@ class Concept:
     exactMatch: Optional[Set[str]] = None
     closeMatch: Optional[Set[str]] = None
     memberOf: Optional[Set[str]] = None
+    remark: Optional[List[Dict[str, str]]] = None
+    range: Optional[List[TextAndURI]] = None
     type: str = "concept"  # used by elasticsearch for indexing
diff --git a/src/fdk_rdf_parser/parse_functions/concept.py b/src/fdk_rdf_parser/parse_functions/concept.py
index 753307d..3b7fda8 100644
--- a/src/fdk_rdf_parser/parse_functions/concept.py
+++ b/src/fdk_rdf_parser/parse_functions/concept.py
@@ -7,6 +7,8 @@ from typing import (
 from rdflib import (
     BNode,
     Graph,
+    Literal,
+    RDF,
     URIRef,
 )
 from rdflib.namespace import (
@@ -57,20 +59,55 @@ def parse_text_and_uri(graph: Graph, subject: URIRef) -> TextAndURI:
     )
 
 
-def extract_range(graph: Graph, definition_ref: URIRef) -> Optional[TextAndURI]:
+def extract_range_deprecated(
+    graph: Graph, definition_ref: URIRef
+) -> Optional[TextAndURI]:
     range_list = []
     for range_ref in graph.objects(definition_ref, skosno_uri("omfang")):
         range_list.append(parse_text_and_uri(graph, range_ref))
     return range_list[0] if len(range_list) == 1 else None
 
 
-def extract_sources(graph: Graph, definition_ref: URIRef) -> Optional[List[TextAndURI]]:
+def extract_range(graph: Graph, definition_ref: URIRef) -> Optional[List[TextAndURI]]:
+    range_list = []
+    for range_obj in graph.objects(definition_ref, skosno_uri("valueRange")):
+        value = TextAndURI()
+        if isinstance(range_obj, URIRef):
+            value.uri = range_obj.toPython()
+        elif isinstance(range_obj, Literal):
+            if range_obj.language:
+                value.text = {range_obj.language: range_obj.toPython()}
+            else:
+                value.text = {"nb": range_obj.toPython()}
+        range_list.append(value)
+    return range_list if len(range_list) > 0 else None
+
+
+def extract_sources_deprecated(
+    graph: Graph, definition_ref: URIRef
+) -> Optional[List[TextAndURI]]:
     sources = []
     for source_ref in graph.objects(definition_ref, DCTERMS.source):
         sources.append(parse_text_and_uri(graph, source_ref))
     return sources if len(sources) > 0 else None
 
 
+def extract_sources(graph: Graph, definition_ref: URIRef) -> Optional[List[TextAndURI]]:
+    sources = []
+    for source_ref in graph.objects(definition_ref, DCTERMS.source):
+        sources.append(
+            TextAndURI(
+                text=None,
+                uri=(
+                    source_ref.toPython()
+                    if source_ref and not isinstance(source_ref, BNode)
+                    else None
+                ),
+            )
+        )
+    return sources if len(sources) > 0 else None
+
+
 def extract_subjects(graph: Graph, concept_ref: URIRef) -> Optional[List[Subject]]:
     subjects = []
     for subject_node in graph.objects(concept_ref, DCTERMS.subject):
@@ -101,7 +138,9 @@ def extract_status(graph: Graph, concept_ref: URIRef) -> Optional[Dict[str, str]
     return status if len(status) > 0 else None
 
 
-def extract_target_group(graph: Graph, definition_ref: URIRef) -> Optional[str]:
+def extract_target_group_deprecated(
+    graph: Graph, definition_ref: URIRef
+) -> Optional[str]:
     uri = object_value(graph, definition_ref, DCTERMS.audience)
     if not uri:
         return None
@@ -113,7 +152,9 @@ def extract_target_group(graph: Graph, definition_ref: URIRef) -> Optional[str]:
         return None
 
 
-def extract_source_relationship(graph: Graph, definition_ref: URIRef) -> Optional[str]:
+def extract_source_relationship_deprecated(
+    graph: Graph, definition_ref: URIRef
+) -> Optional[str]:
     uri = object_value(graph, definition_ref, skosno_uri("forholdTilKilde"))
     if not uri:
         return None
@@ -131,9 +172,22 @@ def parse_definition_deprecated(graph: Graph, definition_ref: URIRef) -> Definit
     return Definition(
         text=value_translations(graph, definition_ref, RDFS.label),
         remark=value_translations(graph, definition_ref, SKOS.scopeNote),
-        targetGroup=extract_target_group(graph, definition_ref),
-        sourceRelationship=extract_source_relationship(graph, definition_ref),
-        range=extract_range(graph, definition_ref),
+        targetGroup=extract_target_group_deprecated(graph, definition_ref),
+        sourceRelationship=extract_source_relationship_deprecated(
+            graph, definition_ref
+        ),
+        range=extract_range_deprecated(graph, definition_ref),
+        sources=extract_sources_deprecated(graph, definition_ref),
+    )
+
+
+def parse_euvoc_definition(graph: Graph, definition_ref: URIRef) -> Definition:
+    return Definition(
+        text=value_translations(graph, definition_ref, RDF.value),
+        targetGroup=object_value(graph, definition_ref, DCTERMS.audience),
+        sourceRelationship=object_value(
+            graph, definition_ref, skosno_uri("relationshipWithSource")
+        ),
         sources=extract_sources(graph, definition_ref),
     )
 
@@ -144,7 +198,11 @@ def parse_skos_definition(graph: Graph, concept_ref: URIRef) -> Definition:
 
 def extract_definition(graph: Graph, concept_uri: URIRef) -> Optional[Definition]:
     definition: Optional[Definition] = None
-    if has_value_on_predicate(graph, concept_uri, SKOS.definition):
+    if has_value_on_predicate(graph, concept_uri, euvoc_uri("xlDefinition")):
+        definition_refs = resource_list(graph, concept_uri, euvoc_uri("xlDefinition"))
+        if len(definition_refs) > 0:
+            definition = parse_euvoc_definition(graph, definition_refs[0])
+    elif has_value_on_predicate(graph, concept_uri, SKOS.definition):
         definition = parse_skos_definition(graph, concept_uri)
     else:
         definition_refs = resource_list(graph, concept_uri, skosno_uri("definisjon"))
@@ -318,4 +376,6 @@ def _parse_concept(
         exactMatch=value_set(graph, concept_uri, SKOS.exactMatch),
         closeMatch=value_set(graph, concept_uri, SKOS.closeMatch),
         memberOf=value_set(graph, concept_uri, uneskos_uri("memberOf")),
+        remark=parse_label_set(graph, concept_uri, SKOS.scopeNote),
+        range=extract_range(graph, concept_uri),
     )


Test patch:
diff --git a/tests/test_concept.py b/tests/test_concept.py
index 12275a9..04df27f 100644
--- a/tests/test_concept.py
+++ b/tests/test_concept.py
@@ -104,11 +104,11 @@ def test_parse_concepts(mock_reference_data_client: Mock) -> None:
                                   skosno:inndelingskriterium  "Inndelingskriterium"@nb ;
                                   xkos:specializes    <http://begrepskatalogen/begrep/bestikk>
                                 ] ;
-        skosno:definisjon   [ a           skosno:Definisjon ;
-                              rdfs:label  "dfgfg"@nb ;
-                              dct:audience skosno:blabla ;
-                              skosno:forholdTilKilde  skosno:blabla
-                            ] .
+        euvoc:xlDefinition      [ a  euvoc:XlNote ;
+                                  rdf:value  "euvoc_definition"@nb ;
+                                  dct:audience <https://data.norge.no/vocabulary/audience-type#public> ;
+                                  skosno:relationshipWithSource <https://data.norge.no/vocabulary/relationship-with-source-type#self-composed>
+                                ] .
 
 <https://registrering-begrep-api.staging.fellesdatakatalog.digdir.no/910258028/3609b02d-72c5-47e0-a6b8-df0a503cf190>
         a                  skos:Concept ;
@@ -279,7 +279,11 @@ def test_parse_concepts(mock_reference_data_client: Mock) -> None:
                 "nb": "gjeldende",
             },
             altLabel=[{"nb": "w"}],
-            definition=Definition(text={"nb": "dfgfg"}),
+            definition=Definition(
+                text={"nb": "euvoc_definition"},
+                targetGroup="https://data.norge.no/vocabulary/audience-type#public",
+                sourceRelationship="https://data.norge.no/vocabulary/relationship-with-source-type#self-composed",
+            ),
             associativeRelation=[
                 AssociativeRelation(
                     description={"nb": "RelationRole"},
@@ -562,7 +566,11 @@ def test_parse_concept(mock_reference_data_client: Mock) -> None:
             skosxl:prefLabel    [ a                   skosxl:Label ;
                                 skosxl:literalForm  "to"@nb
                                 ] ;
-            skos:definition     "skos_definition"@nb .
+            skos:definition     "skos_definition"@nb ;
+            skosno:valueRange
+                                  "test-nn"@nn,
+                                  "test",
+                                  <https://range.com> .
 
 
     <http://publications.europa.eu/resource/authority/concept-status/CURRENT>
@@ -589,6 +597,11 @@ def test_parse_concept(mock_reference_data_client: Mock) -> None:
         uri="https://concepts.staging.fellesdatakatalog.digdir.no/concepts/55a38009-e114-301f-aa7c-8b5f09529f0f",
         created="2023-02-17",
         identifier="https://registrering-begrep-api.staging.fellesdatakatalog.digdir.no/910258028/1843b048-f9af-4665-8e53-3c001d0166c0",
+        range=[
+            TextAndURI(text={"nn": "test-nn"}),
+            TextAndURI(text={"nb": "test"}),
+            TextAndURI(uri="https://range.com"),
+        ],
         harvest=HarvestMetaData(
             firstHarvested="2021-02-17T09:39:13Z", modified="2021-02-17T09:39:13Z"
         ),