Repository: ontodev/robot (java)

Issue:
robot doesn't get pH from current UO by MIREOT
the pH unit is http://purl.obolibrary.org/obo/UO_0000196

It appears twice in the hierarchy, with two different parent classes, and it has also been punned to an individual. 

```Bash
 markampa$ grep http://purl.obolibrary.org/obo/UO_0000196 ontology_downloads/uo.owl 
    <!-- http://purl.obolibrary.org/obo/UO_0000196 -->
    <owl:NamedIndividual rdf:about="http://purl.obolibrary.org/obo/UO_0000196">
                    <rdf:Description rdf:about="http://purl.obolibrary.org/obo/UO_0000196"/>
    <!-- http://purl.obolibrary.org/obo/UO_0000196 -->
    <owl:NamedIndividual rdf:about="http://purl.obolibrary.org/obo/UO_0000196"/>
    <rdf:Description rdf:about="http://purl.obolibrary.org/obo/UO_0000196">
```

But ROBOT doesn't seem to find it with MIREOT:

```Bash
markampa$ robot extract --verbose --method MIREOT --input ./ontology_downloads/uo.owl \
  --lower-term http://purl.obolibrary.org/obo/UO_0000196 --output pH.ttl

markampa$ cat pH.ttl 
@prefix : <http://purl.obolibrary.org/obo/uo.owl#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@base <http://purl.obolibrary.org/obo/uo.owl> .

<http://purl.obolibrary.org/obo/uo.owl> rdf:type owl:Ontology .

###  Generated by the OWL API (version 4.5.6) https://github.com/owlcs/owlapi
```


Pull request description:
Fix for overlapping classes/individuals when running MIREOT extract
Resolves #709

- [x] `docs/` have been added/updated
- [x] tests have been added/updated
- [x] `mvn verify` says all tests pass
- [ ] `mvn site` says all JavaDocs correct
- [x] `CHANGELOG.md` has been updated

When an entity is both a class and an individual, ignore the individual and prefer the class. This required a small hack because if you just call `OntologyHelper.getEntitiy(ontology, iri)` it will not return an entity that occurs as both.


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 69b9eb53..42261170 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Reduced time spent loading datasets for [`query`] in [#666]
 - Fix writing JSON format to use `OutputStream` with ['convert'] in [#671]
 - Fix IRI resolution for `template` in [#689]
+- Fix MIREOT [`extract`] on overlapping class/individual entity for [#709] in [#710]
 - Fix issue with `--add-prefixes` option in [#715]
 
 
@@ -189,6 +190,8 @@ First official release of ROBOT!
 [`template`]: http://robot.obolibrary.org/template
 
 [#715]: https://github.com/ontodev/robot/pull/715
+[#710]: https://github.com/ontodev/robot/pull/710
+[#709]: https://github.com/ontodev/robot/issues/709
 [#689]: https://github.com/ontodev/robot/pull/689
 [#685]: https://github.com/ontodev/robot/pull/685
 [#671]: https://github.com/ontodev/robot/pull/671
diff --git a/docs/extract.md b/docs/extract.md
index 5b15458c..f50c9c82 100644
--- a/docs/extract.md
+++ b/docs/extract.md
@@ -68,6 +68,8 @@ To specify upper and lower term files, use `--upper-terms` and `--lower-terms`.
 
 To only include all descendants of a term or set of terms, use `--branch-from-term` or `--branch-from-terms`, respectively. `--lower-term` or `--lower-terms` are not required when using this option.
 
+Note that if the same IRI is used for both a class and an individual, MIREOT will ignore the individual and only extract the class.
+
 For more details see the [MIREOT paper](http://dx.doi.org/10.3233/AO-2011-0087).
 
 ### Intermediates
diff --git a/robot-core/src/main/java/org/obolibrary/robot/MireotOperation.java b/robot-core/src/main/java/org/obolibrary/robot/MireotOperation.java
index 01306438..03c58d44 100644
--- a/robot-core/src/main/java/org/obolibrary/robot/MireotOperation.java
+++ b/robot-core/src/main/java/org/obolibrary/robot/MireotOperation.java
@@ -4,6 +4,7 @@ import java.util.Collection;
 import java.util.HashSet;
 import java.util.Map;
 import java.util.Set;
+import java.util.stream.Collectors;
 import org.semanticweb.owlapi.apibinding.OWLManager;
 import org.semanticweb.owlapi.model.*;
 import org.semanticweb.owlapi.reasoner.OWLReasoner;
@@ -101,12 +102,19 @@ public class MireotOperation {
     // Get options
     setOptions(options, inputSourceMap);
 
+    // Get all entities in the ontology (preferring Class over NamedIndividual)
+    Set<OWLEntity> entities = getAllEntities(inputOntology);
+
     // The other OWLAPI extract methods use the source ontology IRI
     // so we'll use it here too.
     OWLOntology outputOntology = outputManager.createOntology(inputOntology.getOntologyID());
 
     // Directly copy all upper entities
-    Set<OWLEntity> upperEntities = OntologyHelper.getEntities(inputOntology, upperIRIs);
+    Set<OWLEntity> upperEntities = new HashSet<>();
+    if (upperIRIs != null && upperIRIs.size() > 0) {
+      upperEntities =
+          entities.stream().filter(e -> upperIRIs.contains(e.getIRI())).collect(Collectors.toSet());
+    }
     for (OWLEntity entity : upperEntities) {
       OntologyHelper.copy(inputOntology, outputOntology, entity, annotationProperties);
       if (annotateSource) {
@@ -114,11 +122,13 @@ public class MireotOperation {
       }
     }
 
+    // Create a reasoner to get ancestors
     OWLReasonerFactory reasonerFactory = new StructuralReasonerFactory();
     OWLReasoner reasoner = reasonerFactory.createReasoner(inputOntology);
 
     // For each lower entity, get the ancestors (all or none)
-    Set<OWLEntity> lowerEntities = OntologyHelper.getEntities(inputOntology, lowerIRIs);
+    Set<OWLEntity> lowerEntities =
+        entities.stream().filter(e -> lowerIRIs.contains(e.getIRI())).collect(Collectors.toSet());
     for (OWLEntity entity : lowerEntities) {
       OntologyHelper.copy(inputOntology, outputOntology, entity, annotationProperties);
       if ("none".equals(intermediates)) {
@@ -254,7 +264,11 @@ public class MireotOperation {
     OWLOntologyManager outputManager = OWLManager.createOWLOntologyManager();
     OWLOntology outputOntology = outputManager.createOntology();
 
-    Set<OWLEntity> upperEntities = OntologyHelper.getEntities(inputOntology, upperIRIs);
+    // Get all entities in the ontology (preferring Class over NamedIndividual)
+    Set<OWLEntity> entities = getAllEntities(inputOntology);
+
+    Set<OWLEntity> upperEntities =
+        entities.stream().filter(e -> upperIRIs.contains(e.getIRI())).collect(Collectors.toSet());
     for (OWLEntity entity : upperEntities) {
       OntologyHelper.copy(inputOntology, outputOntology, entity, annotationProperties);
       if ("none".equals(intermediates)) {
@@ -728,6 +742,35 @@ public class MireotOperation {
     }
   }
 
+  /**
+   * Return a set of all OWLEntities from an ontology. If an entity is both an individual and a
+   * class, exclude the indiviudal and only include the class.
+   *
+   * @param inputOntology OWLOntology to get entities from
+   * @return set of OWLEntities
+   */
+  private static Set<OWLEntity> getAllEntities(OWLOntology inputOntology) {
+    Set<OWLEntity> entities = new HashSet<>();
+
+    // Filter out any individuals that have the same IRI as a class (we prefer the class in MIREOT)
+    Set<OWLClass> classes = inputOntology.getClassesInSignature();
+    Set<IRI> classIRIs = classes.stream().map(OWLNamedObject::getIRI).collect(Collectors.toSet());
+    Set<OWLNamedIndividual> individuals = inputOntology.getIndividualsInSignature();
+    individuals =
+        individuals
+            .stream()
+            .filter(i -> !classIRIs.contains(i.getIRI()))
+            .collect(Collectors.toSet());
+
+    entities.addAll(classes);
+    entities.addAll(individuals);
+    entities.addAll(inputOntology.getAnnotationPropertiesInSignature());
+    entities.addAll(inputOntology.getDataPropertiesInSignature());
+    entities.addAll(inputOntology.getObjectPropertiesInSignature());
+
+    return entities;
+  }
+
   /**
    * Given an ontology, a manager for that ontology, an entity to annotate, and a map of source
    * replacements, add the rdfs:isDefinedBy annotation to the entity.
@@ -742,7 +785,10 @@ public class MireotOperation {
     Set<OWLAnnotationValue> existingValues =
         OntologyHelper.getAnnotationValues(ontology, isDefinedBy, entity.getIRI());
     if (existingValues == null || existingValues.size() == 0) {
-      manager.addAxiom(ontology, ExtractOperation.getIsDefinedBy(entity, sourceMap));
+      OWLAnnotationAxiom isDefinedBy = ExtractOperation.getIsDefinedBy(entity, sourceMap);
+      if (isDefinedBy != null) {
+        manager.addAxiom(ontology, isDefinedBy);
+      }
     }
   }
 


Test patch:
diff --git a/robot-core/src/test/java/org/obolibrary/robot/MireotOperationTest.java b/robot-core/src/test/java/org/obolibrary/robot/MireotOperationTest.java
index f74e3e66..fb017d52 100644
--- a/robot-core/src/test/java/org/obolibrary/robot/MireotOperationTest.java
+++ b/robot-core/src/test/java/org/obolibrary/robot/MireotOperationTest.java
@@ -24,7 +24,25 @@ public class MireotOperationTest extends CoreTest {
    */
   @Test
   public void testMireot() throws Exception {
-    testMireot("/mireot.owl");
+    testMireot(
+        "/filtered.owl",
+        "/mireot.owl",
+        "http://purl.obolibrary.org/obo/uberon.owl",
+        "http://purl.obolibrary.org/obo/UBERON_0001235");
+  }
+
+  /**
+   * Test MIROET on a subset with an entity that is both a class and individual.
+   *
+   * @throws Exception
+   */
+  @Test
+  public void testMireotOnOverlap() throws Exception {
+    testMireot(
+        "/pH.ttl",
+        "/pH-extract.ttl",
+        "http://purl.obolibrary.org/obo/uo.owl",
+        "http://purl.obolibrary.org/obo/UO_0000196");
   }
 
   /**
@@ -33,13 +51,11 @@ public class MireotOperationTest extends CoreTest {
    * @param expectedPath the path to a known-good file for comparison
    * @throws Exception on any problem
    */
-  public void testMireot(String expectedPath) throws Exception {
-    OWLOntology inputOntology = loadOntology("/filtered.owl");
-
-    IRI outputIRI = IRI.create("http://purl.obolibrary.org/obo/uberon.owl");
-
-    Set<IRI> upperIRIs =
-        Collections.singleton(IRI.create("http://purl.obolibrary.org/obo/UBERON_0001235"));
+  public void testMireot(String inputPath, String expectedPath, String outputIRIString, String term)
+      throws Exception {
+    OWLOntology inputOntology = loadOntology(inputPath);
+    IRI outputIRI = IRI.create(outputIRIString);
+    Set<IRI> upperIRIs = Collections.singleton(IRI.create(term));
     // Set<IRI> branchIRIs = upperIRIs;
 
     List<OWLOntology> outputOntologies = new ArrayList<>();
diff --git a/robot-core/src/test/resources/pH-extract.ttl b/robot-core/src/test/resources/pH-extract.ttl
new file mode 100644
index 00000000..9a38133c
--- /dev/null
+++ b/robot-core/src/test/resources/pH-extract.ttl
@@ -0,0 +1,33 @@
+@prefix : <http://purl.obolibrary.org/obo/uo.owl#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix xml: <http://www.w3.org/XML/1998/namespace> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@base <http://purl.obolibrary.org/obo/uo.owl> .
+
+<http://purl.obolibrary.org/obo/uo.owl> rdf:type owl:Ontology .
+
+#################################################################
+#    Annotation properties
+#################################################################
+
+###  http://www.w3.org/2000/01/rdf-schema#comment
+rdfs:comment rdf:type owl:AnnotationProperty .
+
+
+###  http://www.w3.org/2000/01/rdf-schema#label
+rdfs:label rdf:type owl:AnnotationProperty .
+
+
+#################################################################
+#    Classes
+#################################################################
+
+###  http://purl.obolibrary.org/obo/UO_0000196
+<http://purl.obolibrary.org/obo/UO_0000196> rdf:type owl:Class ;
+                                            rdfs:comment "\"A dimensionless concentration notation which denotes the acidity of a solution in terms of activity of hydrogen ions (H+).\" [Wikipedia:Wikipedia]"^^xsd:string ;
+                                            rdfs:label "pH"^^xsd:string .
+
+
+###  Generated by the OWL API (version 4.5.6) https://github.com/owlcs/owlapi
diff --git a/robot-core/src/test/resources/pH.ttl b/robot-core/src/test/resources/pH.ttl
new file mode 100644
index 00000000..32fceda1
--- /dev/null
+++ b/robot-core/src/test/resources/pH.ttl
@@ -0,0 +1,74 @@
+@prefix : <http://purl.obolibrary.org/obo/uo.owl#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix xml: <http://www.w3.org/XML/1998/namespace> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@base <http://purl.obolibrary.org/obo/uo.owl> .
+
+<http://purl.obolibrary.org/obo/uo.owl> rdf:type owl:Ontology .
+
+#################################################################
+#    Annotation properties
+#################################################################
+
+###  http://www.w3.org/2000/01/rdf-schema#comment
+rdfs:comment rdf:type owl:AnnotationProperty .
+
+
+###  http://www.w3.org/2000/01/rdf-schema#label
+rdfs:label rdf:type owl:AnnotationProperty .
+
+
+#################################################################
+#    Classes
+#################################################################
+
+###  http://purl.obolibrary.org/obo/UO_0000000
+<http://purl.obolibrary.org/obo/UO_0000000> rdf:type owl:Class ;
+                                            rdfs:subClassOf owl:Thing ;
+                                            rdfs:comment "\"A unit of measurement is a standardized quantity of a physical quality.\" [Wikipedia:Wikipedia]"^^xsd:string ;
+                                            rdfs:label "unit"^^xsd:string .
+
+
+###  http://purl.obolibrary.org/obo/UO_0000051
+<http://purl.obolibrary.org/obo/UO_0000051> rdf:type owl:Class ;
+                                            rdfs:subClassOf <http://purl.obolibrary.org/obo/UO_0000000> ;
+                                            rdfs:comment "\"A unit which represents a standard measurement of how much of a given substance there is mixed with another substance.\" [UOC:GVG]"^^xsd:string ;
+                                            rdfs:label "concentration unit"^^xsd:string .
+
+
+###  http://purl.obolibrary.org/obo/UO_0000186
+<http://purl.obolibrary.org/obo/UO_0000186> rdf:type owl:Class ;
+                                            rdfs:subClassOf <http://purl.obolibrary.org/obo/UO_0000000> ;
+                                            rdfs:comment "\"A unit which is a standard measure of physical quantity consisting of only a numerical number without any units.\" [Wikipedia:Wikipedia]"^^xsd:string ;
+                                            rdfs:label "dimensionless unit"^^xsd:string .
+
+
+###  http://purl.obolibrary.org/obo/UO_0000196
+<http://purl.obolibrary.org/obo/UO_0000196> rdf:type owl:Class ;
+                                            rdfs:subClassOf <http://purl.obolibrary.org/obo/UO_0000051> ,
+                                                            <http://purl.obolibrary.org/obo/UO_0000186> ,
+                                                            <http://purl.obolibrary.org/obo/UO_1000196> ;
+                                            rdfs:comment "\"A dimensionless concentration notation which denotes the acidity of a solution in terms of activity of hydrogen ions (H+).\" [Wikipedia:Wikipedia]"^^xsd:string ;
+                                            rdfs:label "pH"^^xsd:string .
+
+
+###  http://purl.obolibrary.org/obo/UO_1000196
+<http://purl.obolibrary.org/obo/UO_1000196> rdf:type owl:Class ;
+                                            rdfs:subClassOf <http://purl.obolibrary.org/obo/UO_0000051> ,
+                                                            <http://purl.obolibrary.org/obo/UO_0000186> ;
+                                            rdfs:label "pH based unit"^^xsd:string .
+
+
+#################################################################
+#    Individuals
+#################################################################
+
+###  http://purl.obolibrary.org/obo/UO_0000196
+<http://purl.obolibrary.org/obo/UO_0000196> rdf:type owl:NamedIndividual ;
+rdfs:comment "\"A dimensionless concentration notation which denotes the acidity of a solution in terms of activity of hydrogen ions (H+).\" [Wikipedia:Wikipedia]"^^xsd:string ;
+                                            rdfs:label "pH"^^xsd:string .
+
+
+###  Generated by the OWL API (version 4.5.9.2019-02-01T07:24:44Z) https://github.com/owlcs/owlapi