Repository: explosion/spaCy

Issue:
Error should be raised if you try to set conflicting doc.ents
## How to reproduce the behaviour
```python
import spacy
from spacy.tokens import Span
nlp = spacy.load('en_core_web_sm')
doc = nlp(u"Louisiana Office of Conservation")
print(doc.ents[0], doc.ents[0].label_)

entity = Span(doc, 0, 1, label=391L)
print(entity, entity.label_)

doc.ents = list(doc.ents) + [entity]

print(doc.ents[0], doc.ents[0].label_)
```
## Output:
`(Louisiana Office of Conservation, u'ORG')`
`(Louisiana, u'MONEY')`
`(Louisiana Office of Conservation, u'MONEY')`

## Expected Behaviour:
I was expecting MONEY entity to override the ORG entity and end offset of MONEY should have been used after merging. Please let me know if I got this wrong.

## Your Environment
* Operating System:Windows 7
* Python Version Used: 2.7.12
* spaCy Version Used: 2.0.7
* Environment Information: None



Discussion:
Thanks a lot for the great and detailed bug report 👍  I just tried the example and I can confirm that it's also reproducible in v2.1.0 nightly. The current behaviour is definitely a bug.

In terms of the expected behaviour: I suggest that spaCy should actually raise an error here. By definition, a token can only be part of *one* entity. So when the `doc.ents` are set and an overlapping entity exists, there's no clear answer for how this should be resolved. Which entity should take precedence? 

So it's probably more intuitive and logical if spaCy raised with a message like: "Trying to set conflicting `doc.ents`: `(0, 4, 'ORG')` and `(0, 1, 'MONEY')`. A token can only be part of one entity, so make sure the entities you're setting don't overlap."
Thanks for quick reply Ines. I think the new span can take preference on existing one as this will help users to create new derived entities.

Resolving patch:
diff --git a/spacy/errors.py b/spacy/errors.py
--- a/spacy/errors.py
+++ b/spacy/errors.py
@@ -250,7 +250,9 @@ class Errors(object):
     E096 = ("Invalid object passed to displaCy: Can only visualize Doc or "
              "Span objects, or dicts if set to manual=True.")
     E097 = ("Can't merge non-disjoint spans. '{token}' is already part of tokens to merge")
-
+    E098 = ("Trying to set conflicting doc.ents: '{span1}' and '{span2}'. A token"
+            " can only be part of one entity, so make sure the entities you're "
+            "setting don't overlap.")
 
 @add_codes
 class TempErrors(object):


Test patch:
diff --git a/spacy/tests/doc/test_add_entities.py b/spacy/tests/doc/test_add_entities.py
--- a/spacy/tests/doc/test_add_entities.py
+++ b/spacy/tests/doc/test_add_entities.py
@@ -3,6 +3,7 @@
 
 from ...pipeline import EntityRecognizer
 from ..util import get_doc
+from ...tokens import Span
 
 import pytest
 
@@ -22,3 +23,13 @@ def test_doc_add_entities_set_ents_iob(en_vocab):
 
     doc.ents = [(doc.vocab.strings['WORD'], 0, 2)]
     assert [w.ent_iob_ for w in doc] == ['B', 'I', '', '']
+
+def test_add_overlapping_entities(en_vocab):
+    text = ["Louisiana", "Office", "of", "Conservation"]
+    doc = get_doc(en_vocab, text)
+    entity = Span(doc, 0, 4, label=391)
+    doc.ents = [entity]
+
+    new_entity = Span(doc, 0, 1, label=392)
+    with pytest.raises(ValueError):
+        doc.ents = list(doc.ents) + [new_entity]
diff --git a/spacy/tests/regression/test_issue242.py b/spacy/tests/regression/test_issue242.py
--- a/spacy/tests/regression/test_issue242.py
+++ b/spacy/tests/regression/test_issue242.py
@@ -17,9 +17,10 @@ def test_issue242(en_tokenizer):
     matcher.add('FOOD', None, *patterns)
 
     matches = [(ent_type, start, end) for ent_type, start, end in matcher(doc)]
-    doc.ents += tuple(matches)
     match1, match2 = matches
     assert match1[1] == 3
     assert match1[2] == 5
     assert match2[1] == 4
     assert match2[2] == 6
+
+    doc.ents += tuple([match2])