fils/oihBook huggingface dataset:
Source and Prov Approaches#
About#
This section will present an identifer discussion and then focus
on approaches publishers can use to provide more source and provenance
information.This section is heavily influenced
by [1] [2]  and the reader is highly
encouraged to read these references.
It is not uncommon for a single resource to be described at multiple locations.
The following items represent properties that can help address the
disambiguation of resources as the Ocean InfoHub graph grows.
We will look at the following seven properties:
identifier, provider, publisher, sameAs, isBasesOn, subjectOf and its
inverse, about.
It should be noted that only subjectOf and sameAs can be used on type
Thing.  That is, these properties can be used on any Schema.org type.
The others have their domain and range values listed.  All these
properties work on CreativeWork. So they are all valid for that type
and subtypes like Dataset and Map.
Example Graph#
The following example graph shows some of the properties we can use
to provide source and provenance information about a resource.
 1{
 2    "@context": {
 3        "@vocab": "https://schema.org/"
 4    },
 5    "@id": "https://example.org/id/XYZ",
 6    "@type": "CreativeWork",
6    "@type": "CreativeWork",
 7    "url": "https://example.org/url/of/the/item",
 8    "sameAs": "https://doi.org/10.5066/F7VX0DMQ",
 9    "isBasedOn": "https://example.org/id/xyz",
10    "identifier": {
11        "@id": "https://doi.org/10.5066/F7VX0DMQ",
12        "@type": "PropertyValue",
13        "propertyID": "https://registry.identifiers.org/registry/doi",
14        "url": "https://doi.org/10.5066/F7VX0DMQ",
15        "value": "doi:10.5066/F7VX0DMQ"
16    },
17    "subjectOf": {
18        "@type": "CreativeWork",
19        "@id": "http://purl.unep.org/sdg/SDGIO_00020173",
20        "url": "http://www.ontobee.org/ontology/SDGIO?iri=http://purl.unep.org/sdg/SDGIO_00020173",
21        "name": "UNSD SDG indicator code:C140c01",
21        "name": "UNSD SDG indicator code:C140c01",
22        "description": "Number of countries making progress ... the oceans and their resources"
23    },
24    "provider": {
25        "@id": "https://www.sample-data-repository.org",
26        "@type": "Organization",
27        "legalName": "Sample Data Repository Office",
28        "name": "SDRO",
29        "sameAs": "http://www.re3data.org/repository/r3dxxxxxxxxx",
30        "url": "https://www.sample-data-repository.org"
31    },
32    "publisher": {
33        "@id": "https://www.sample-data-repository.org"
34    }
35}
import json
from pyld import jsonld
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
    doc = json.load(dgraph)
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
jbutils.show_graph(compacted)
compacted = jsonld.compact(doc, context)
jbutils.show_graph(compacted)
identifier#
This is the main subject of the start of this section.  Please
refer there for details on this property.
Values expected to be one of these types
PropertyValue
Text
URL
Used on these types
Thing
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "CreativeWork",
  "identifier": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/XYZ",
    "@type": "CreativeWork",
    "identifier": {
"@type": "CreativeWork",
    "identifier": {
        "@id": "https://doi.org/10.5066/F7VX0DMQ",
        "@type": "PropertyValue",
        "propertyID": "https://registry.identifiers.org/registry/doi",
        "url": "https://doi.org/10.5066/F7VX0DMQ",
        "value": "doi:10.5066/F7VX0DMQ"
    }
}
propertyID#
A commonly used identifier for the characteristic represented by the property, e.g. a manufacturer or a standard code for a property. propertyID can be (1) a prefixed string, mainly meant to be used with standards for product properties; (2) a site-specific, non-prefixed string (e.g. the primary key of the property or the vendor-specific id of the property), or (3) a URL indicating the type of the property, either pointing to an external vocabulary, or a Web resource that describes the property (e.g. a glossary entry). Standards bodies should promote a standard prefix for the identifiers of properties from their standards.
value#
The value of the quantitative value or property value node. For PropertyValue,
it can be ‘Text;’, ‘Number’, ‘Boolean’, or ‘StructuredValue’.
url (pointing to type URL)#
URL of the item.
url (pointing to type URL)#
URL of the item.
provider#
schema.org/provider
The service provider, service operator, or service performer; the goods
producer. Another party (a seller) may offer those services or goods on behalf
of the provider. A provider may also serve as the seller.
For OIH this is the agent that is responsible for distributing the resource
and the descriptive metadata.  That is, the provider actually runs and supports
the services that presents the resource on the net or otherwise makes the
data available.
Values expected to be one of these types
Organization
Person
Used on these types
CreativeWork
EducationalOccupationalProgram
Invoice
ParcelDelivery
Reservation
Service
Trip
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
    doc = json.load(dgraph)
frame = {
doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "CreativeWork",
  "provider": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/XYZ",
    "@type": "CreativeWork",
    "provider": {
"@type": "CreativeWork",
    "provider": {
        "@id": "https://www.sample-data-repository.org",
        "@type": "Organization",
        "legalName": "Sample Data Repository Office",
        "name": "SDRO",
        "sameAs": "http://www.re3data.org/repository/r3dxxxxxxxxx",
        "url": "https://www.sample-data-repository.org"
    }
}
publisher#
See: schema.org/publisher
The publisher is defined as “The publisher of the creative work”.  This is
viewed as the agent that is primarily responsible for making the content described
by the structured metadata.
Values expected to be one of these types
Organization
Person
Used on these types
CreativeWork
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "CreativeWork",
  "publisher": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/XYZ",
    "@type": "CreativeWork",
    "publisher": {
"@type": "CreativeWork",
    "publisher": {
        "@id": "https://www.sample-data-repository.org",
        "@type": "Organization",
        "legalName": "Sample Data Repository Office",
        "name": "SDRO",
        "sameAs": "http://www.re3data.org/repository/r3dxxxxxxxxx",
        "url": "https://www.sample-data-repository.org"
    }
}
sameAs#
See: schema.org/sameAs
The sameAs property links a Thing to a URL.  It is expected that the
URL is a resource that is the the most canonical URL for the original.
In cases where your resource is not the canonical URL, you can use the
sameAs property to link to the canonical URL.  This is useful when you wish
to publish a resource and give credit to the original resource.  Note,
in this case the resource must not be altered, but actually be the same
as the canonical URL resource.
The sameAs property should always point to only one resource.  It is not
logically consistent to point to multiple sameAs resources.
Values expected to be one of these types
URL
Used on these types
Thing
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
    doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "CreativeWork",
  "sameAs": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/XYZ",
    "@type": "CreativeWork",
    "sameAs": "https://doi.org/10.5066/F7VX0DMQ"
}
isBasedOn#
See: schema.org/isBasedOn
Where sameAs is used to link to the canonical URL of the resource, isBasedOn
provides a means to link derivative works to the original resource this new
resources is based on.
The isBasedOn property can be used to link to multiple resources if more
than one was used in the generation of this new resource.
Values expected to be one of these types
CreativeWork
Product
URL
Used on these types
CreativeWork
subjectOf and inverse about#
See: schema.org/subjectOf
The property subjectOf of can be used to indicate a resource that is related to
the described resource, although not necessarily a part of it. The subjectOf
property can be used in an educational framework to indicate the field(s) of
science or literature the dataset relates to.
Values expected to be one of these types
CreativeWork
Event
Values expected to be one of these types
CreativeWork
Event
Used on these types
Thing
The subjectOf property has an inverse-property about.
The property about can be used to:
indicate the subject matter this thing is about
Values expected to be one of these types
Thing
Used on these types
CreativeWork
Event
CommunicateAction
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/identifier/graphs/creativework.json") as dgraph:
    doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "CreativeWork",
  "subjectOf": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/XYZ",
    "@type": "CreativeWork",
    "subjectOf": {
        "@id": "http://purl.unep.org/sdg/SDGIO_00020173",
        "@type": "CreativeWork",
        "description": "Number of countries making progress ... the oceans and their resources",
        "name": "UNSD SDG indicator code:C140c01",
"name": "UNSD SDG indicator code:C140c01",
        "url": "http://www.ontobee.org/ontology/SDGIO?iri=http://purl.unep.org/sdg/SDGIO_00020173"
    }
}
References#
1
Omar Benjelloun, Shiyu Chen, and Natasha Noy, editors. Google Dataset Search by the Numbers, 2020. URL: https://arxiv.org/abs/2006.06894.
2
Google search central dataset. https://developers.google.com/search/docs/advanced/structured-data/dataset. URL: https://developers.google.com/search/docs/advanced/structured-data/dataset.
Linking to documents and resources#
Leveraging the ability to link between resources can serve many goals.  We may
wish to demonstrate connections between people and courses they have taken or
or organizations they are connected with.   We may be wishing to link documents
to people or organizations.
This section will review two key thematic profiles and some examples of how to
express links from them to other resources.   Our goal will be different in various
cases.  The two profiles are type CreativeWork and type Organization.
In the case of Organization our purpose may be to express alignment to various
principles and policies.  These might provide people with an understanding of
the goals of an organization when they are searching for or assessing them.
In the case of CreativeWork we are looking to express connections to the
publisher and provider of the creative work.   This is mostly to connect these
works with the responsible party associated with them but may also serve to
connect to the principles they are associated with.
Organization link options#
In the following section we will look at three different options for expressing
links between an organization and resources that describe the policy and
principles of the subject organization.
First we will see the full data graph.  We have highlighted the sections we
we will review here.  Namely the subjectOF and publishingPrinciples
predicates.
 1{
 2    "@context": {
predicates.
 1{
 2    "@context": {
 3        "@vocab": "https://schema.org/"
 4    },
 5    "@id": "https://example.org/id/org/x",
 6    "@type": "Organization",
 7    "address": {
 8        "@type": "PostalAddress",
 9        "addressLocality": "Paris, France",
10        "postalCode": "F-75002",
11        "streetAddress": "38 avenue de l'Opera"
12    },
13    "email": "secretariat(at)example.org",
14    "name": "Organization X",
15    "description": "Description of org ...",
16    "telephone": "( 33 1) 42 68 53 00",
17    "member": [
18        {
19            "@type": "Organization",
20            "name": "Organization A",
21            "description": "Org A is a potential parent organization of Org X"
22        }
23    ],
24    "identifier": {
25        "@id": "https://grid.ac/institutes/grid.475727.4",
26        "@type": "PropertyValue",
27        "description": "UN Department of Economic and Social Affairs Sustainable Development",
27        "description": "UN Department of Economic and Social Affairs Sustainable Development",
28        "propertyID": "https://registry.identifiers.org/registry/grid",
29        "url": "https://grid.ac/institutes/grid.475727.4"
30    },
31    "subjectOf": {
32        "@type": "CreativeWork",
33        "@id": "http://purl.unep.org/sdg/SDGIO_00020173",
34        "url": "http://www.ontobee.org/ontology/SDGIO?iri=http://purl.unep.org/sdg/SDGIO_00020173",
35        "name": "UNSD SDG indicator code:C140c01",
36        "description": "Number of countries making progress ... the oceans and their resources"
37    },
38    "publishingPrinciples": [
39        {
40            "@type": "CreativeWork",
41            "@id": "https://sdgs.un.org/goals/goal14",
42            "url": "https://sdgs.un.org/goals/goal14",
43            "name": "Sustainable Development Goal 14",
44            "description": "Conserve and sustainably use the oceans, seas and marine resources for sustainable development"
45        },
46        {
45        },
46        {
47            "@type": "CreativeWork",
48            "@id": "https://dx.doi.org/10.1038/sdata.2016.18",
49            "url": "https://www.nature.com/articles/sdata201618",
50            "name": "FAIR data principles",
51            "description": "FAIR Principles definition as referenced from: Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship."
52        }
53    ]
54}
import json
from pyld import jsonld
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/sdg/graphs/org.json") as dgraph:
    doc = json.load(dgraph)
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
jbutils.show_graph(compacted)
subjectOf#
Values expected to be one of these types
Event
CreativeWork
Used on these types
Thing
Lets take a look at subjectOf.  In this case we are using subjectOf to express
a connection to a UN SDG.  This, subjectOf, could also be used to connect
documents describing the policy and principles of an organization or additional
metadata for a creative work.  When we look at subjectOf
we can see we are allowed are allowed to use it on any type Thing, but must point
to a CreativeWork or Event.
Note
Recall that in the case of OIH types, the type CourseInstance or EducationEvent are both
subtype of Event.  Given that we can use subjectOf to connect a Thing to these types
as well.  Also, Course is a subtype of CreativeWork, so we are good there too in the
context of the range of subjectOf.   Reference thematic type Training
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/sdg/graphs/org.json") as dgraph:
    doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "Organization",
  "subjectOf": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/org/x",
    "@type": "Organization",
    "subjectOf": {
        "@id": "http://purl.unep.org/sdg/SDGIO_00020173",
        "@type": "CreativeWork",
        "description": "Number of countries making progress ... the oceans and their resources",
        "name": "UNSD SDG indicator code:C140c01",
        "url": "http://www.ontobee.org/ontology/SDGIO?iri=http://purl.unep.org/sdg/SDGIO_00020173"
    }
}
publishingPrinciples#
Values expected to be one of these types
CreativeWork
URL
Used on these types
CreativeWork
Organization
Person
Lets take a look at publishingPrinciples.  This can be used
to connect CreativeWork, Organization, or Person to either of CreativeWork or URL.  So this
allows us to link a CreativeWork to a policy or principle statement.  This has some very useful
use cases where resources can be grouped based on their connection to those principles and policies.
import json
from rdflib.extras.external_graph_libs import rdflib_to_networkx_multidigraph
from rdflib.extras.external_graph_libs import rdflib_to_networkx_graph
from pyld import jsonld
import graphviz
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils
with open("../../../odis-in/dataGraphs/thematics/sdg/graphs/org.json") as dgraph:
    doc = json.load(dgraph)
frame = {
  "@context": {"@vocab": "https://schema.org/"},
  "@explicit": "true",
  "@requireAll": "true",
  "@type":     "Organization",
  "publishingPrinciples": ""
}
context = {
    "@vocab": "https://schema.org/",
}
compacted = jsonld.compact(doc, context)
framed = jsonld.frame(compacted, frame)
jd = json.dumps(framed, indent=4)
print(jd)
jbutils.show_graph(framed)
print(jd)
jbutils.show_graph(framed)
{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@id": "https://example.org/id/org/x",
    "@type": "Organization",
    "publishingPrinciples": [
        {
            "@id": "https://sdgs.un.org/goals/goal14",
            "@type": "CreativeWork",
            "description": "Conserve and sustainably use the oceans, seas and marine resources for sustainable development",
            "name": "Sustainable Development Goal 14",
            "url": "https://sdgs.un.org/goals/goal14"
        },
        {
            "@id": "https://dx.doi.org/10.1038/sdata.2016.18",
            "@type": "CreativeWork",
"@type": "CreativeWork",
            "description": "FAIR Principles definition as referenced from: Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship.",
            "name": "FAIR data principles",
            "url": "https://www.nature.com/articles/sdata201618"
        }
    ]
}
Sustainable Development Goals#
The following example provides an an approach to connecting
Sustainable Development Goals
(SDGs) could be linked to via subjectOf.
Other potential links could be made to things such
as the UNDRR-ISC Hazard Definition & Classification
As this is a CreateWork, we can now use one more linking property, the
Schema.org citation property.  By comparison, the
publishingPrinciples or
subjectOf
connections carry a bit more semantic meaning.
 1{
 2  "@context": {
 3    "@vocab": "https://schema.org/"
 4  },
 5  "@type": "CreativeWork",
 6  "@id": "https://example.org/id/XYZ",
 7  "name": "Name or title of the document",
 8  "description": "Description of the resource to aid in searching",
 9  "distribution": {
10    "@type": "DataDownload",
9  "distribution": {
10    "@type": "DataDownload",
11    "contentUrl": "https://www.sample-data-repository.org/dataset/472032.tsv",
12    "encodingFormat": "text/tab-separated-values"
13  },
14  "citation": {
15    "@type": "CreativeWork",
16    "@id": "http://purl.unep.org/sdg/SDGIO_00020173",
17    "url": "http://www.ontobee.org/ontology/SDGIO?iri=http://purl.unep.org/sdg/SDGIO_00020173",
18    "name": "UNSD SDG indicator code:C140c01",
19    "description": "Number of countries making progress ... the oceans and their resources"
20  },
21  "maintainer" : {
22     "@type" : "Organization",
23     "@id": "https://ror.org/050bms902",
24     "description": "UN Department of Economic and Social Affairs Sustainable Development"
25  }
26}
import json
from pyld import jsonld
import os, sys
currentdir = os.path.dirname(os.path.abspath(''))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
from lib import jbutils