Repository: flairNLP/flair

Issue:
Direct access to Token tags removed on version 0.11
```
from flair.data import Sentence
from flair.models import SequenceTagger

tagger = SequenceTagger.load("flair/ner-dutch-large")

sentence = Sentence("George Washington ging naar Washington")

tagger.predict(sentence)

for token in sentence:
 print(token.text, token.get_tag("ner"))
```

the above code gives the corresponding named entity to each token in a sentence. After upgrading to v0.11 the method get_tag has been removed and I can't find a similar method that will give similar results.  I checked get_labels but for some reason, I am getting an empty list.

Thank you for your help.


Discussion:
You have to replace 

```
token.get_tag("ner")
```
by 

```
token.get_label("ner")
```
However, I have found that all labels are now set to 'O' in some cases regardless of the detected entities. However, if you do

```
spans = sentence.get_spans("ner")
```
it works as expected and returns the right spans.

Any reason why this new behavior happens? I think it may be related to [#2728](https://github.com/flairNLP/flair/issues/2728)


Hello @MoncefBenaicha and @melayadi yes it is the same reason as #2728: Span annotations are now directly predicted as such instead of being broken down into individual Token labels. So from the point of view of the end user, there are on longer the labels 'B-PER', 'I-PER', 'S-PER' stored at Tokens, but rather a single label 'PER' that is stored for the entire span. 

The reason for this is twofold: 
- one the one hand, we assume most users just care about getting the full entity ("George Washington") and not be interested in the BIOES tags that led to the prediction
- one the other hand, not all NER methods actually use BIOES tags (for instance, biaffine approaches directly predict spans). We are now adding different NER approaches to Flair and want to keep the output consistent across all. In this case, that means that all NER approaches should in the end predict spans. 

As written in #2728, you now get the NER labels by doing: 

```python
tagger = SequenceTagger.load('ner-fast')

sentence = Sentence("George Washington lives in New York")

tagger.predict(sentence)

# print entities
for entity in sentence.get_spans('ner'):
    print(entity) 
```

You can access lots of fields on the `Span` object, check the tutorial [here](https://github.com/flairNLP/flair/blob/master/resources/docs/TUTORIAL_2_TAGGING.md#getting-annotated-spans). 

If you want to have `Token` annotations, you could transfer the labels to the Token level with this code: 

```python
tagger = SequenceTagger.load('ner-fast')

sentence = Sentence("George Washington lives in New York")

tagger.predict(sentence)

# transfer entity labels to token level
for entity in sentence.get_spans('ner'):
    prefix = 'B-'
    for token in entity:
        token.set_label('ner-bio', prefix + entity.tag, entity.score)
        prefix = 'I-'

# now go through all tokens and print label
for token in sentence:
    print(token)
``` 
Thanks for response, makes sense. If I have multiple tagged sentences in a list and I'd like to save them in ConLL format, is there a Flair functionality of doing that directly? Or I have to set the IOB(ES) token labels as you mnetioned and write the tokens to the file as a for-loop?
You can use any BIO(ES) (etc) format you desire. The real issue is that the scores are lost on each of the tags. This is a reduction in functionality in the least - we consider it a bug in 11.x. It is keeping us from adopting the latest version of Flair and I don't believe we will be able to move forward with Flair in the long term if this is not resolved. I can't imagine that we are the only customer you have which wants the Flair neural network to return both the predicted labels and scores per tag in the span.

I have responded by email previously with a solution and will be happy to provide it in this forum. Please contact me at jenorton@firstam.com for more details.
Hello,
getting the actual tag probabilities is still possible when running `sentence_tagger.predict(sentence, return_probabilities_for_all_classes=True)` and can be accessed via `sentence.get_tags_proba_dist(sequence_tagger.tag_type)`
however the output is currently not nice, I proposed a fix in this pr: https://github.com/flairNLP/flair/pull/2731
Thank you for your immediate attention to this manner.

Get_tags_proba_dist is on the tagger.

Problem is I found another bug.  When I try to print the label (my model gives me 20 to look at), I get the following crash:

  File "/lib/python3.8/site-packages/flair/data.py", line 234, in __str__
    return f"{self.data_point.unlabeled_identifier}{flair._arrow}{self._value} ({round(self._score, 4)})"
AttributeError: 'str' object has no attribute 'unlabeled_identifier'

That should probably be fixed as well.

Jeff Norton

From: BenediktFuchs ***@***.***>
Sent: Saturday, April 16, 2022 1:22 AM
To: flairNLP/flair ***@***.***>
Cc: Jeffrey Norton ***@***.***>; Comment ***@***.***>
Subject: [External] Re: [flairNLP/flair] Direct access to Token tags removed on version 0.11 (Issue #2729)


Hello,
getting the actual tag probabilities is still possible when running sentence_tagger.predict(sentence, return_probabilities_for_all_classes=True) and can be accessed via sentence.get_tags_proba_dist(sequence_tagger.tag_type)
however the output is currently not nice, I proposed a fix in this pr: #2731<https://urldefense.com/v3/__https:/github.com/flairNLP/flair/pull/2731__;!!L1aKtqoz4WY!YGZxS1iF-67hvoLAOV4eJ_lqZra4CVvj_smo7upfzwO7SCth1hFgRY7X3KBXSfR1uwwucKYolDfYCbs1MozfwLxC$>

—
Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/flairNLP/flair/issues/2729*issuecomment-1100605183__;Iw!!L1aKtqoz4WY!YGZxS1iF-67hvoLAOV4eJ_lqZra4CVvj_smo7upfzwO7SCth1hFgRY7X3KBXSfR1uwwucKYolDfYCbs1Mk9_Og8u$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AYV3FRZCAQWGN3XMKLVCS7LVFJ2CRANCNFSM5TLN5SQQ__;!!L1aKtqoz4WY!YGZxS1iF-67hvoLAOV4eJ_lqZra4CVvj_smo7upfzwO7SCth1hFgRY7X3KBXSfR1uwwucKYolDfYCbs1MjBLu79_$>.
You are receiving this because you commented.Message ID: ***@***.******@***.***>>

******************************************************************************************
This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. 
If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. 
If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter.

If you received this email as a commercial message and would like to opt out of future commercial messages, please let us know and we will remove you from our distribution list.

Thank you.
******************************************************************************************
FAFLD


Resolving patch:
diff --git a/flair/models/sequence_tagger_model.py b/flair/models/sequence_tagger_model.py
--- a/flair/models/sequence_tagger_model.py
+++ b/flair/models/sequence_tagger_model.py
@@ -424,7 +424,8 @@ def predict(
         label_name: Optional[str] = None,
         return_loss=False,
         embedding_storage_mode="none",
-    ):
+        force_token_predictions: bool = False,
+    ):  # type: ignore
         """
         Predicts labels for current batch with CRF or Softmax.
         :param sentences: List of sentences in batch
@@ -506,7 +507,7 @@ def predict(
                 for sentence, sentence_predictions in zip(batch, predictions):
 
                     # BIOES-labels need to be converted to spans
-                    if self.predict_spans:
+                    if self.predict_spans and not force_token_predictions:
                         sentence_tags = [label[0] for label in sentence_predictions]
                         sentence_scores = [label[1] for label in sentence_predictions]
                         predicted_spans = get_spans_from_bio(sentence_tags, sentence_scores)


Test patch:
diff --git a/tests/test_sequence_tagger.py b/tests/test_sequence_tagger.py
--- a/tests/test_sequence_tagger.py
+++ b/tests/test_sequence_tagger.py
@@ -14,7 +14,7 @@
 
 @pytest.mark.integration
 def test_load_use_tagger():
-    loaded_model: SequenceTagger = SequenceTagger.load("ner")
+    loaded_model: SequenceTagger = SequenceTagger.load("ner-fast")
 
     sentence = Sentence("I love Berlin")
     sentence_empty = Sentence("       ")
@@ -22,22 +22,32 @@ def test_load_use_tagger():
     loaded_model.predict(sentence)
     loaded_model.predict([sentence, sentence_empty])
     loaded_model.predict([sentence_empty])
-    del loaded_model
 
-    sentence.clear_embeddings()
-    sentence_empty.clear_embeddings()
+    # check that "Berlin" is predicted as LOC-Span
+    assert len(sentence.get_spans("ner")) == 1
+    assert sentence.get_spans("ner")[0].text == "Berlin"
+    assert sentence.get_spans("ner")[0].tag == "LOC"
+
+    # check that "Berlin" is predicted as S-LOC-Token when force_token_predictions=True
+    sentence = Sentence("I love Berlin")
+    loaded_model.predict(sentence, force_token_predictions=True)
+    assert sentence.get_token(3).text == "Berlin"
+    assert sentence.get_token(3).tag == "S-LOC"
+
+    del loaded_model
 
     loaded_model: SequenceTagger = SequenceTagger.load("pos")
 
     loaded_model.predict(sentence)
     loaded_model.predict([sentence, sentence_empty])
     loaded_model.predict([sentence_empty])
+
     del loaded_model
 
 
 @pytest.mark.integration
 def test_load_use_tagger_keep_embedding():
-    loaded_model: SequenceTagger = SequenceTagger.load("ner")
+    loaded_model: SequenceTagger = SequenceTagger.load("ner-fast")
 
     sentence = Sentence("I love Berlin")
     loaded_model.predict(sentence)
@@ -53,7 +63,7 @@ def test_load_use_tagger_keep_embedding():
 
 @pytest.mark.integration
 def test_all_tag_proba_embedding():
-    loaded_model: SequenceTagger = SequenceTagger.load("ner")
+    loaded_model: SequenceTagger = SequenceTagger.load("ner-fast")
 
     sentence = Sentence("I love Berlin")
     loaded_model.predict(sentence, return_probabilities_for_all_classes=True)