Run run_tf_glue.py has bugs
Author: ltang666
Pull request: no

do as https://github.com/huggingface/transformers/tree/master/examples/text-classification suggests, but has following bugs showing **/home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory**

my dataset_info.json copy from https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/testing/metadata/glue/cola/1.0.0/dataset_info.json

`06/13/2020 14:21:11 - INFO - absl -   Field info.citation from disk and from code do not match. Keeping the one from code.
06/13/2020 14:21:11 - INFO - absl -   Field info.location from disk and from code do not match. Keeping the one from code.
06/13/2020 14:21:11 - INFO - absl -   Reusing dataset glue (/home/admin/tensorflow_datasets/glue/cola/1.0.0)
06/13/2020 14:21:11 - INFO - absl -   Constructing tf.data.Dataset for split train, from /home/admin/tensorflow_datasets/glue/cola/1.0.0
dddddddddddddd: <PrefetchDataset shapes: {idx: (), label: (), sentence: ()}, types: {idx: tf.int32, label: tf.int64, sentence: tf.string}>
2020-06-13 14:21:12.132176: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at iterator_ops.cc:929 : Not found: /home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory
Traceback (most recent call last):
  File "./examples/text-classification/run_tf_glue.py", line 229, in <module>
    main()
  File "./examples/text-classification/run_tf_glue.py", line 175, in main
    if training_args.do_train
  File "./examples/text-classification/run_tf_glue.py", line 55, in get_tfds
    return glue_convert_examples_to_features(ds, tokenizer, max_seq_length, task_name)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 62, in glue_convert_examples_to_features
    return _tf_glue_convert_examples_to_features(examples, tokenizer, max_length=max_length, task=task)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 79, in _tf_glue_convert_examples_to_features
    examples = [processor.tfds_map(processor.get_example_from_tensor_dict(example)) for example in examples]
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 79, in <listcomp>
    examples = [processor.tfds_map(processor.get_example_from_tensor_dict(example)) for example in examples]
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 622, in __next__
    return self.next()
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 666, in next
    return self._next_internal()
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 651, in _next_internal
    output_shapes=self._flat_output_shapes)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_dataset_ops.py", line 2673, in iterator_get_next_sync
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: /home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory [Op:IteratorGetNextSync]
run.sh: line 15: 15230 Segmentation fault      (core dumped) CUDA_VISIBLE_DEVICES=2 python ./examples/text-classification/run_tf_glue.py --model_name_or_path bert-base-uncased --task_name $TASK_NAME --do_train --do_eval --max_seq_length 12 --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /home/admin/test/transformers/tmp/$TASK_NAME/
`

Comments: