Exception: EOF while parsing a string at line 1 column 8862550
Author: MrAsimZahid
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.20.1 also tried with 4.15.0
- Platform: Linux-5.13.0-48-generic-x86_64-with-glibc2.31
- Python version: 3.9.12
- Huggingface_hub version: 0.5.1
- PyTorch version (GPU?): 1.11.0+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@SaulLu @LysandreJik

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the behavior:

1. ```git clone https://github.com/aiforsec/CyNER.git```
2.  Go into the cloned repository folder
3. ```pip install -r requirements.txt```
4. open ```CyNER Demo.ipynb``` notebook
5. Run first and the very last cell for training.

Getting results like this:
https://github.com/aiforsec/CyNER/issues/4


### Expected behavior

The model starts training properly and completes training.

Comments:
Hi @MrAsimZahid ,

I followed the first 4 steps to try to reproduce your problem. Unfortunately I get the error bellow (I have spacy installed so it seems that it is necessary to be more restrictive on the versions in the requirements file - see my environment [here](https://github.com/huggingface/transformers/files/9041404/env.txt)):
```bash
Traceback (most recent call last):
  File "notebook.py", line 5, in <module>
    import cyner
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/__init__.py", line 1, in <module>
    from .cyner import CyNER
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/cyner.py", line 1, in <module>
    from .entity_extraction_factory import EntityExtractionFactory as eef
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/entity_extraction_factory.py", line 3, in <module>
    from .spacy_ner import Spacy
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/spacy_ner.py", line 1, in <module>
    import spacy
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/spacy/__init__.py", line 15, in <module>
    from .cli.info import info  # noqa: F401
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/spacy/cli/__init__.py", line 17, in <module>
    from .debug_diff import debug_diff  # noqa: F401
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/spacy/cli/debug_diff.py", line 10, in <module>
    from .init_config import init_config, Optimizations
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/spacy/cli/init_config.py", line 8, in <module>
    from jinja2 import Template
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/jinja2/defaults.py", line 3, i
[env.txt](https://github.com/huggingface/transformers/files/9041403/env.txt)
n <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/jinja2/filters.py", line 13, in <module>
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/site-packages/markupsafe/__init__.py)
```

If you ever find a way to isolate the bug related to transformers from the rest of your software solution, it could surely help us to help you more quickly. :hugs:


 

Hi @SaulLu,

Thank you for taking the time to review the bug.

I am using this
```pip install MarkupSafe==2.0.1```

I still have some issues: NLTK wasn't installed and now I have:
```bash
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/lucile_huggingface_co/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/lucile_huggingface_co/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/lucile_huggingface_co/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/lucile_huggingface_co/anaconda3/envs/cyner/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/lucile_huggingface_co/repos/CyNER/notebook.py", line 48, in <module>
    entities = model3.get_entities(text)
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/cyner.py", line 60, in get_entities
    entities = model.get_entities(text)
  File "/home/lucile_huggingface_co/repos/CyNER/cyner/flair_ner.py", line 24, in get_entities
    for x in pred['entities']:
KeyError: 'entities'
```

I'm sorry, I don't really have time to look for (maybe obvious) solutions to make this notebook work. As said before, it would help me a lot if you could isolate the problem you are having with transformers. :slightly_smiling_face: 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.