Token-classification, pipeline leaving out answers
Author: stulcrad-ctu
Pull request: no
Labels: name: Core: Pipeline

name: Good Difficult Issue

### System Info

`transformers` version: 4.36.2
- Platform: Linux-4.18.0-372.26.1.el8_6.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.4
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.26.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2+cu121 (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

This line of code
```
model = AutoModelForTokenClassification.from_pretrained("stulcrad/CNEC2_0_Supertypes_xlm-roberta-large")
tokenizer = AutoTokenizer.from_pretrained("stulcrad/CNEC2_0_Supertypes_xlm-roberta-large")
text = "Titanik je historicka loď, která se potopila v roce 1912. Potopila se v Atlantiku."
ner = pipeline("ner", model=model, tokenizer=tokenizer)
ner(text)
```
outputs
```
[{'entity': 'B-O',
  'score': 0.9997367,
  'index': 1,
  'word': '▁Titan',
  'start': 0,
  'end': 5},
 {'entity': 'I-O',
  'score': 0.9995271,
  'index': 2,
  'word': 'ik',
  'start': 5,
  'end': 7},
 {'entity': 'B-T',
  'score': 0.9999306,
  'index': 16,
  'word': '▁1912',
  'start': 52,
  'end': 56},
 {'entity': 'B-G',
  'score': 0.9999751,
  'index': 23,
  'word': '▁Atlantik',
  'start': 72,
  'end': 80},
 {'entity': 'I-G',
  'score': 0.99998987,
  'index': 24,
  'word': 'u',
  'start': 80,
  'end': 81}]
```
However, the same line of code, but with aggregation_strategy="simple" in pipeline outputs
```
[{'entity_group': 'T',
  'score': 0.9999306,
  'word': '1912',
  'start': 52,
  'end': 56},
 {'entity_group': 'G',
  'score': 0.9999825,
  'word': 'Atlantiku',
  'start': 72,
  'end': 81}]
```


### Expected behavior

Aggregation_strategy set to simple should output
```
[{'entity_group': 'O',
  'score': ...,
  'word': 'Titanik',
  'start': 0,
  'end': 7},
{'entity_group': 'T',
  'score': 0.9999306,
  'word': '1912',
  'start': 52,
  'end': 56},
 {'entity_group': 'G',
  'score': 0.9999825,
  'word': 'Atlantiku',
  'start': 72,
  'end': 81}]
```

Comments:
Hey! I am not sure I quite understand the issue here, not super familiar with this pipeline. 
Leaving it to the community to find a cool fix! 🤗 

> Hey! I am not sure I quite understand the issue here, not super familiar with this pipeline. Leaving it to the community to find a cool fix! 🤗

I am sorry, I will try to specify it a little more.

The entity Titanik should be placed in an entity group of "O", which in my dataset is an entity for artifact names, pipeline picks it up, but it leaves out the answer for some reason, when the aggregation strategy is set, to simplify the output. The same thing happens on the hugging face site for the model.

Description of the entities - https://huggingface.co/stulcrad/CNEC_extended_xlm-roberta-large

For aggregation_strategy="simple", we are first grouping (and setting the entity_group parameter, which is 'O' for the 'artifact name entity': 'Titanik'), then we are filtering the entity_group 'O' which is 'Outside of a named entity'. Since both have same tag (which is 'O'), the word 'Titanik' is getting dropped. 
@stulcradCVUT You can use different tag (anything other than 'O') for the 'artifact name entity' and it should work.
@ArthurZucker If you think that this is a bug, then I can take it up.

does not sound like a bug, thanks for explaning! 

Hey! Is this issue still active? @RVV-karma’s explanation seems pretty reasonable. If the issue persists and hasn’t been resolved yet, I would be happy to work on it and help find a solution.


> Hey! Is this issue still active? @RVV-karma’s explanation seems pretty reasonable. If the issue persists and hasn’t been resolved yet, I would be happy to work on it and help find a solution.

Sorry, I forgot to close it,  the explanation was enough, just a little change in that naming was all I needed.