text: 
  
  TITLE
  ValueError("The training dataset must have an asserted cardinality") when running run_tf_ner.py
  
  COMMENTS
  5
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  - `transformers` version: 4.2.0
  - Platform: linux
  - Python version: python3.6
  - PyTorch version (GPU?): 1.7.1 gpu
  - Tensorflow version (GPU?):2.4.0
  - Using GPU in script?: yes
  - Using distributed or parallel set-up in script?:no
  
  ### Who can help
   @stefan-it
   
  ## Information
  
  Model I am using (Bert, XLNet ...): bert-base-multilingual-cased
  
  The problem arises when using:
  
  - [yes ] the official example scripts: (give details below)
  
  * [ ] my own modified scripts: (give details below)
  
  The tasks I am working on is:
  * [ ] an official GLUE/SQUaD task: (give the name) ner  GermEval 2014 
  * [ ] my own task or dataset: (give details below)
  
  ## To reproduce
  
  Steps to reproduce the behavior:
  
  1. Created a new conda environment using conda env -n xftf2 python=3.6
  2. pip insrall transformer==4.2.0 tensorflow==2.4 torch==1.7.1
  3. Prepare the data set(train.txt, test.txt, dev.txt) according to the README under the folder token-classification, run run_tf_ner.py 
     setting  from_pt=True, with the following parameters:
  ```
  --data_dir ./data \
  --labels ./data/labels.txt \
  --model_name_or_path bert-base-multilingual-cased \
  --output_dir ./output \
  --max_seq_length  128 \
  --num_train_epochs 4\
  --per_device_train_batch_size 32 \
  --save_steps 500 \
  --seed 100 \
  --do_train \
  --do_eval \
  --do_predict
  ```
  Here is the stack trace:
  ```
  01/21/2021 00:12:18 - INFO - utils_ner -   *** Example ***
  01/21/2021 00:12:18 - INFO - utils_ner -   guid: dev-5
  01/21/2021 00:12:18 - INFO - utils_ner -   tokens: [CLS] Dara ##us entwickelte sich im Rok ##oko die Sitt ##e des gemeinsamen Wein ##ens im Theater , das die Stand ##es ##grenze ##n innerhalb des Publikum ##s über ##brücken sollte . [SEP]
  01/21/2021 00:12:18 - INFO - utils_ner -   input_ids: 101 95621 10251 28069 10372 10211 51588 20954 10128 105987 10112 10139 58090 90462 12457 10211 16223 117 10242 10128 15883 10171 58433 10115 21103 10139 63332 10107 10848 99765 17799 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  01/21/2021 00:12:18 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  01/21/2021 00:12:18 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  01/21/2021 00:12:18 - INFO - utils_ner -   label_ids: -1 24 -1 24 24 24 6 -1 24 24 -1 24 24 24 -1 24 24 24 24 24 24 -1 -1 -1 24 24 24 -1 24 -1 24 24 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
  Traceback (most recent call last):
    File "run_tf_ner.py", line 299, in <module>
      main()
    File "run_tf_ner.py", line 231, in main
      trainer.train()
    File "/.conda/envs/xftf2/lib/python3.6/site-packages/transformers/trainer_tf.py", line 457, in train
      train_ds = self.get_train_tfdataset()
    File "/.conda/envs/xftf2/lib/python3.6/site-packages/transformers/trainer_tf.py", line 141, in get_train_tfdataset
      raise ValueError("The training dataset must have an asserted cardinality")
  ValueError: The training dataset must have an asserted cardinality
  ```
  
  ## Expected behavior
  In such a case, is there any tips to deal with it?I really appreciate any help you can provide.
  